Do IMF fiscal forecasts add value?

• Published date: 01 September 2018
• Date: 01 September 2018
• DOI: http://doi.org/10.1002/for.2527

RESEARCH ARTICLE

Do IMF fiscal forecasts add value?

Zidong An

1,3

| João Tovar Jalles

2

| Prakash Loungani

3

| Ricardo M. Sousa

4,5

1

College of Arts and Sciences, American

University, Washington, DC, USA

2

Fiscal Affairs Department, International

Monetary Fund, Washington, DC, USA

3

Research Department, International

Monetary Fund, Washington, DC, USA

4

Department of Economics and Economic

Policies Research Unit (NIPE), University

of Minho, Braga, Portugal

5

LSE Alumni Association, London School

of Economics and Political Science,

London, UK

Correspondence

João Tovar Jalles, Research Department,

International Monetary Fund, 700 19th

Street NW, Washington, DC, 20431, USA.

Email: jjalles@imf.org

Abstract

We used a panel of 29 advanced and emerging market countries to investigate

whether the IMF's World Economic Outlook (WEO) fiscal forecasts add value

in terms of forecast accuracy and information content, relative to private sector

forecasts (from Consensus Economics). We find that: (i) WEO forecasts are not

significantly less accurate than Consensus forecasts; (ii) WEO and Consensus

forecasts tend to mutually encompass one another; and (iii) each source of

forecasts appears to contain some information that is not embedded in the

other source.

KEYWORDS

asymmetry, bias, budget balance, efficiency, forecast comparison, information rigidity

1|INTRODUCTION

Understanding budgetary dynamics is a very important

as well as a timely and timeless question. Forecasting it

correctly, however, seems an equally important issue,

particularly in a context where social expenditure is ris-

ing and fiscal policy action is limited in many countries

given the continued public finance retrenchment. While

the literature on forecast performance broadly supports

the view that fiscal forecasts tend to be biased and inac-

curate (Jonung & Larch, 2006), assessing the perfor-

mance of fiscal forecasts emanating from different

sources for a heterogeneous panel of advanced and

emerging market countries has not been sufficiently cov-

ered in the literature. In fact, it is well known that gov-

ernments are often subject to pressures that lead to the

production of budget balance forecasts that are too opti-

mistic (Frankel, 2012). A fair witness of the skepticism

that typically surrounds fiscal forecasts by local authori-

ties is assembled in several studies in the literature that

confirm the lack of accuracy of such forecasts (Leal,

Perez, Tujula, & Vidal, 2008). For instance, Frankel

(2011) shows that the actual fiscal balance outcomes

are systematically lower than the official forecasts

produced by governments; that is, there is evidence of a

significantly positive bias in fiscal forecasts.

This empirical observation raises a number of rele-

vant questions. Do fiscal forecasts produced by interna-

tional organizations outperform those of private sector

agencies? Can national government forecasts be

complemented by different sources of forecasts? Is there

any information content in forecasts from strengthened

national institutions or supranational organizations that

is not embedded in private sector forecasts?

In this paper, we analyze these issues through the lens

of a rigorous econometric assessment. More specifically,

we start by highlighting that while standard measures,

such as the mean forecast error (ME), the mean absolute

error (MAE), or the root mean squared error (RMSE), are

valuable when evaluating the forecast performance of a

given institution (Artis & Marcellino, 2001; Keereman,

1999), they are much less informative about the relative

performance of “competing”sources of forecasts. For this

reason, we directly test the significance of differences in

mean square errors between: (i) the IMF's World Eco-

nomic Outlook (WEO) fiscal forecasts (i.e., an

Received: 7 April 2017 Revised: 12 December 2017 Accepted: 2 April 2018

DOI: 10.1002/for.2527

650 Copyright © 2018 John Wiley & Sons, Ltd. Journal of Forecasting. 2018;37:650–665.wileyonlinelibrary.com/journal/for

international organization); and (ii) the fiscal forecasts of

Consensus Economics (i.e., a private‐sector agency).

We accomplish such a goal by relying on two major

tests. The first one is test developed by Ashley, Granger,

and Schmalensee (1980), which regresses the differences

in errors from two forecasts on the sums of these errors.

This allows us to compare the accuracy of WEO forecasts

vis‐à‐vis Consensus Economics forecasts. The second test

follows the procedure put forward by Fair and Shiller

(1990), and aims at assessing whether one forecast domi-

nates the other one in terms of information content.

Therefore, by combining the information content of two

sets of forecasts into a single “combined”forecast, the test

makes it possible to investigate whether each forecast

contains information that is not already embedded in

the other forecast.

Using data for a panel of 29 advanced and emerging

market countries over the period 1993–2014, we find that

WEO forecasts are not significantly less accurate than

Consensus forecasts. Additionally, WEO and Consensus

forecasts tend to mutually encompass one another. This

suggests that a combined forecast might yield better

results. Moreover, each source of forecasts appears to

contain some information that is not already included

in the other source.

Our results also confirm the bias toward “optimism”

that typically characterizes fiscal forecasts. We show

that Consensus forecasts tend to be more optimistic that

WEO forecasts, especially for the subsample of emerging

market economies where WEO forecasts are actually pes-

simistic. Despite this, traditional metrics of forecast accu-

racy suggest that WEO forecasts outperform Consensus

forecasts both across the time horizon dimension and

country groups.

In addition, we find that WEO forecasts are more effi-

cient than Consensus forecasts, with the latter generally

being inefficient. In the case of emerging markets, the

lack of forecast efficiency accrues to the fact that past

errors are repeated in the present. For advanced econo-

mies, inefficiency also reflects that the model from which

forecasts are generated is not a minimum‐variance

model. The empirical results also convey the message that

inefficiency is still observed once we take asymmetry into

account. Indeed, both positive and negative past forecast

errors are correlated with present forecast errors.

Finally, we show that information rigidity is a feature

of Consensus forecasts, but less so for WEO forecasts:

new information tends to be incorporated more quickly

in the revised forecasts of the latter than the former.

The rest of the paper is organized as follows. Section 2

describes the data. Section 3 presents the econometric

methodology. Section 4 discusses the empirical results.

Finally, Section 5 concludes.

2|DATA

We consider a panel dataset consisting of 29 countries

(including advanced and emerging market economies)

between 2001 and 2014.

1

Our dependent variable is the

forecast for the budget balance‐to‐GDP ratio, and we rely

on the projections made by WEO and Consensus

Economics.

In the case of the WEO, it has published semi‐annual

forecasts over the period 2001–2014. In both cases, the

event being forecasted is the average budget balance‐to‐

GDP ratio for a given target year. We index the sequence

of forecasts by the horizon, H. As for Consensus Eco-

nomics, it has published monthly forecasts for major eco-

nomic variables and an increasing number of countries

on a monthly basis since 1989. For each country, the

majority of the forecasters are from the private sector,

with the number of forecasters varying between about

10 and 30. The “consensus”forecast is based on the

arithmetic mean of the forecasts, and we gather informa-

tion for forecasts of the budget balance for the period

January 2001–December 2014.

2

Figure 1 plots the distribution of forecasts of the budget

balance‐to‐GDP ratios for advanced and emerging coun-

tries over the sample period. We present the distribution

for horizons h= 20 (21), h= 14 (15), h= 8 (9), and h=2

(3) for WEO (Consensus). The Consensus is treated com-

paratively in an advantageous way; that is, for both cur-

rent‐year and year‐ahead forecasts, the forecast horizon

for Consensus is 1 month later relative to WEO's.

3

At h= 20, h= 21, the distribution for both WEO and

Consensus forecasts, respectively, is centered around a

deficit of between 3% and 4% of gross domestic product

(GDP) in the case of advanced economies, but it is wider

for emerging markets (between 1% and 5% of GDP). As

the forecast horizon shortens, the distribution of forecasts

1

The list of countries included in the analysis is: advanced economies—

Australia, Canada, France, Germany, Italy, Japan, New Zealand, UK,

and USA; emerging market economies—Argentina, Brazil, Bulgaria,

China, Chile, Croatia, Czech Republic, Estonia, Hong Kong SAR, Hun-

gary, India, Latvia, Lithuania, Mexico, Poland, Russia, Slovenia, Taiwan

POC, Turkey, and Venezuela.

2

Note that Consensus started publishing forecasts of the budget balance

in 1993, but for reasons related to the perfect overlap with WEO fore-

casts we only rely on those starting in 2001.

3

Most Consensus forecasts are released at the beginning of a given

month, meaning that the information set available to private forecasters

stops 1 month before. This gives Consensus forecasts an implicit disad-

vantage by construction. Against this background, we decided to give

Consensus forecasts a 1‐month advantage since we are trying to argue

that, even with 1‐month disadvantage, WEO forecasts still add value.

Note that we have also conducted our empirical analysis by giving Con-

sensus zero advantage (i.e., using the same months for both sources) and

our results hold.

AN ET AL.651