Backtesting Value‐at‐Risk: A Generalized Markov Test

• Date: 01 August 2017
• Published date: 01 August 2017
• Author: Thor Pajhede
• DOI: http://doi.org/10.1002/for.2456

Journal of Forecasting,J. Forecast. 36, 597–613 (2017)

Published online 6 February 2017 in Wiley Online Library (wileyonlinelibrary.com)DOI: 10.1002/for.2456

Backtesting Value-at-Risk: A Generalized Markov Test

Thor Pajhede1,2

1

Department of Economics, University of Copenhagen, Denmark

2

CREATES, University of Aarhus, Denmark

ABSTRACT

Testing the validity of value-at-risk (VaR) forecasts, or backtesting, is an integral part of modern market risk man-

agement and regulation. This is often done by applying independence and coverage tests developed by Christoffersen

(International Economic Review, 1998; 39(4), 841–862) to so-called hit-sequences derived from VaR forecasts and

realized losses. However, as pointed out in the literature, these aforementioned tests suffer from low rejection fre-

quencies, or (empirical) power when applied to hit-sequences derived from simulations matching empirical stylized

characteristics of return data. One key observation of the studies is that higher-order dependence in the hit-sequences

may cause the observed lower power performance. We propose to generalize the backtest framework for VaRforecasts,

by extending the original first-order dependence of Christoffersen to allow for a higher- or kth-order dependence. We

provide closed-form expressions for the tests as well as asymptotic theory.Not only do the generalized tests have power

against kth-order dependence by definition, but also included simulations indicate improved power performance when

replicating the aforementioned studies. Further, included simulations show much improved size properties of one of

the suggested tests. Copyright © 2017 John Wiley & Sons, Ltd.

KEY WORDS value-at-risk; backtesting; Markov chain; duration; quantile; likelihood ratio; maximum

likelihood

INTRODUCTION

Since its introduction in the 1990s value-at-risk (VaR), as measured by the pth quantile of a forecasted distribution

of losses, has become widely used when reporting aggregate market risk. This again has prompted a rich literature on

validation of VaR forecasts, so-called backtesting, as is often applied empirically by regulatory authorities, academics,

and financial institutions. See Campbell (2007) for a review of the backtesting procedures and an economic motivation

for the backtesting criteria.

The leading reference on backtesting is Christoffersen (1998), wherein the evaluation of accurate Va R forecasts

was first formalized. Specifically it was shown that the occurrences of losses beyond a specified VaR level, termed

violations or hits, should occur independently and with a constant probability matching the pth quantile. Based on

this, the widely applied conditional coverage and independence tests were proposed. However, as documented in

Christoffersen and Pelletier (2004) and Berkowitz et al. (2011), the tests have low empirical power in simulation

studies matching empirical stylized facts of returns data.

To address this we propose to derive tests in a more general setting than the original framework of Christoffersen

(1998). Specifically, we propose tests within a general backtest frameworkextending the underlying Markovian model

of Christoffersen to allow for higher- or kth-order dependence. Withinthe quite general kth-order dependence model,

we consider two structures or specifications: one which we label as the generalized Markov specification, and the

other as the generalized Markov duration specification. Preceding the details given in Section 2, the generalized

Markov specification can be viewed as similar to the extension of autoregressive models from order one to order k

when testing for white noise, while the Markov duration specification mimics the duration modeling approaches to

backtesting of Christoffersen and Pelletier (2004), Haas (2006), and Pelletier and Wei (2016).

We provide asymptotic theory and closed-form expressions for the implied tests for conditional coverage and

independence within these generalized specifications. Moreover, simulations illustrate that the new generalized tests

solve some of the leading issues with regard to low empirical power.

Note, in this respect, that by definition the proposed tests will have power against higher-order dependence, and in

particular so when compared to the tests derived in the Markovian framework. That the tests seem to perform well in

empirically stylized simulations is an additional reason to prefer these.

The rest of the paper is organized as follows. Section 2 sets out the backtesting criteria, that is, unconditional

coverage, independence, and conditional coverage. Section 2 reviews the popular classic Markov backtests due to

Christoffersen (1998) and Kupiec (1995). Section 2 introduces our new framework. We consider two specifications

Correspondence to: Thor Pajhede, Department of Economics, University of Copenhagen, Øster Farimagsgade 5, 1353 2 Copenhagen K,

Denmark. E-mail: thor.nielsen@econ.ku.dk

Copyright © 2017 John Wiley & Sons, Ltd

598 T. Pajhede

from this framework: the generalized Markov and the Markov duration specifications. From them we derive tests of

unconditional coverage, independence and conditional coverage. Section 2 examines the power and size properties of

the various tests using a simulation framework. Section 2 concludes.

HIT-SEQUENCE-BASED BACKTESTING

Let Rtdenote the realization of a return of an asset or a portfolio of assets at time t. The ex ante VaR for time t

and coverage rate p, denoted as VaRtjt1.p/, conditional on all information, Ft1, available at time t1(e.g., past

returns and macroeconomic indicators) is defined as the pth conditional quantile of the distribution of Rt:

P.R

t<VaR tjt1.p /jFt1/Dp; t D1;:::;T: (1)

Typically, the coverage rate used is 1%or5%. Several parametric (e.g., generalized autoregressive heteroskedas-

ticity [GARCH] models) and nonparametric (e.g., historical simulation) methods are used to forecast VaRtjt1.p/

(McNeil et al., 2005).

Backtesting is the procedure of comparing realized losses to the forecasted VaR. To implement backtesting of a

VaR forecast, we follow Christoffersen (1998) in defining the hit-sequence, ¹ItºT

tD1, as follows:

Definition 1. The hit-sequence, ¹ItºT

tD1, for a sequence of VaR forecasts, ®VaRtjt1.p /¯T

tD1,isdefinedas

ItWD 1Rt<VaR tjt1.p /;tD1;:::;T; (2)

where 1./is the indicator function. Thus the hit-sequence is by construction a binary time series indicating whether

a loss at time tgreater than the VaR, termed a violation or a hit, was realized.

A VaR forecast is valid, in the sense of actually having forecasted the desired quantile, only if the associated

hit-sequence satisfies the following criteria due to Christoffersen (1998):

The unconditional coverage criterion. The unconditional probability of a violation must be exactly equal to the

coverage rate p:

HUCWP.I

tD1/ Dp: (3)

The independence criterion. The conditional probability of a violation must be constant:

HIndWP.I

tD1jFt1/DP.I

tD1/: (4)

Combining these criteria, we obtain the conditional coverage criterion:

The conditional coverage criterion. The probability of a violation must be constant and equal to the coverage rate:

HCCWP.I

tD1jFt1/DP.I

tD1/ Dp: (5)

It follows (see Christoffersen, 1998) that the hit-sequence of a valid VaR forecast i s in fact a sequence of i.i.d.

Bernoulli distributed variables:

It

i.i.d. Bernoulli.p/; t D1;:::;T: (6)

The classic Markov framework of Christoffersen (1998) models the hit-sequence of Equation (6) as a first-order

Markov chain. As detailed in Section 2 below,this allows testing of both the unconditional coverage and independence

criteria using likelihood ratio tests. Furthermore, these tests have closed-form expressions, standard asymptotics, and

are easy to implement. However, as previously mentioned in the introduction, the tests have also been found to suffer

from low power when dependence is not Markovian of order one.

In Section 2, we extend the classic Markov framework to allow for higher- or kth-order dependence. We detail

how our approach preserves all of the aforementioned advantages of the classic Markov testing, but also has power

against more general forms of dependence.

Classic Markov testing

The first backtest by Kupiec (1995) models the hit-sequence as an i.i.d. Bernoulli sequence with an unknown

probability parameter 120; 1Œ,thatis,

It

i.i.d. Bernoulli.1/; t D1;:::;T: (7)

Copyright © 2017 John Wiley & Sons, Ltd J. Forecast. 36, 597–613 (2017)