A likelihood ratio and Markov chain‐based method to evaluate density forecasting

• Published date: 01 January 2020
• Author: Jonas Andersson,Yushu Li
• Date: 01 January 2020
• DOI: http://doi.org/10.1002/for.2604

RESEARCH ARTICLE

A likelihood ratio and Markov chain‐based method to

evaluate density forecasting

Yushu Li

1,2

| Jonas Andersson

3

1

Department of Mathematics, University

of Bergen, Bergen, Norway

2

Department of Economics and Statistics,

Linnaeus University, Småland, Sweden

3

Department of Business and

Management Science, Norwegian School

of Economics, Bergen, Norway

Correspondence

Yushu Li, Department of Mathematics,

University of Bergen, Norway.

Email: yushu.li@uib.no

Funding information

Finance Market Fund, Norwegian

Research Council, Grant/Award Number:

274569

Abstract

In this paper, we propose a likelihood ratio‐based method to evaluate density

forecasts, which can jointly evaluate the unconditional forecasted distribution

and dependence of the outcomes. Unlike the well‐known Berkowitz test, the

proposed method does not require a parametric specification of time dynamics.

We compare our method with the method proposed by several other tests and

show that our methodology has very high power against both dependence and

incorrect forecasting distributions. Moreover, the loss of power, caused by the

nonparametric nature of the specification of the dynamics, is shown to be small

compared to the Berkowitz test, even when the parametric form of dynamics is

correctly specified in the latter method.

KEYWORDS

density forecasting, likelihood ratio test, Markov chain

1|INTRODUCTION

An evaluation of the quality of forecasts can have differ-

ent purposes. It could be to determine whether point fore-

casts are, on average, hitting the actual outcome not yet

observed. It could be, for example, in a risk management

context, to investigate whether interval forecasts have the

coverage probability the model used would imply. The

evaluation of point forecasts is typically done by compar-

ing different forecasting models and investigating

whether one has a significantly larger expected loss func-

tion. This loss function could be mean squared error

(MSE), mean absolute error (MAE) or, in cases where

available, economic loss incurred by using a forecast com-

pared to having the actual values. Examples on papers

dealing with the evaluation of point forecasts are Wallis

(1995), Diebold and Lopez (1996), and Gneiting (2011).

Interval forecasts are evaluated by the relative frequency

of an interval to cover the actual outcome (Chatfield,

1993; Granger, White, & Kamstra, 1989). An often cited

paper on the evaluation of interval forecasting is

Christoffersen (1998), which proposed a theory to evalu-

ate the interval forecast. This evaluation procedure is

based on the likelihood ratio test and, owing to the addi-

tivity of the likelihood ratio test, the method can jointly

test the unconditional coverage and independence by

testing the correct conditional coverage. This test and its

extensions (Berkowitz, Christoffersen, & Pelletier, 2011;

Clements & Taylor, 2003; Dumitrescu, Hurlin, &

Madkour, 2013; Engle & Manganelli, 2004) are most

widely used to evaluate an interval forecast, especially

in the value‐at‐risk (VaR) analysis, which can be viewed

as a one‐sided interval forecast.

Finally, an even more detailed forecast is the density

forecast, which estimates the probability density of a

future value of the process, conditional on the

------------------------------------------------------- -- --- -- -- --- -- --- -- -- --- -- --- -- -- --- -- --- -- -- --- -- --- -- --- -- -- --- -- --- -- -

This is an open access article under the terms of the Creative Commons Attribution‐NonCommercial License, which permits use, distribution and reproduction in any

medium, provided the original work is properly cited and is not used for commercial purposes.

© 2019 The Authors Journal of Forecasting Published by John Wiley & Sons Ltd

Received: 7 January 2013 Revised: 24 April 2018 Accepted: 3 May 2019

DOI: 10.1002/for.2604

Journal of Forecasting. 2020;39:47–55. wileyonlinelibrary.com/journal/for 47