The impact of parameter and model uncertainty on market risk predictions from GARCH‐type models

• Date: 01 November 2017
• Author: David Ardia,Jeremy Kolly,Denis‐Alexandre Trottier
• Published date: 01 November 2017
• DOI: http://doi.org/10.1002/for.2472

Received: 31 October 2016 Revised: 2 March 2017 Accepted: 3 March 2017

DOI: 10.1002/for.2472

RESEARCH ARTICLE

The impact of parameter and model uncertainty on market risk

predictions from GARCH-type models

David Ardia1,2 Jeremy Kolly2,3 Denis-Alexandre Trottier2

1Institute of Financial Analysis, University

of Neuchâtel, Neuchâtel, Switzerland

2Finance, Insurance and Real Estate

Department, Laval University,Québec,

Canada

3Department of Management, University of

Fribourg, Fribourg, Switzerland

Correspondence

Jeremy Kolly,Depar tment of Management,

University of Fribourg, Boulevard de

Pérolles 90, CH-1700 Fribourg, Switzerland.

Email: jeremy.kolly@unifr.ch

Funding information

Swiss National Science Foundation, Grant

Number: 158754;FQRSC, Grant Number:

2015-NP-179931

Abstract

We study the effect of parameter and model uncertainty on the left-tail of predictive

densities and in particular on VaR forecasts. To this end, we evaluate the predictive

performance of several GARCH-type models estimated via Bayesian and maxi-

mum likelihood techniques. In addition to individual models, several combination

methods are considered, such as Bayesian model averaging and (censored) optimal

pooling for linear, log or beta linear pools. Daily returns for a set of stock market

indexes are predicted overabout 13 years from the early 2000s. We find that Bayesian

predictive densities improve the VaR backtest at the 1% risk level for single models

and for linear and log pools. We also find that the robust VaR backtest exhibited by

linear and log pools is better than the backtest of single models at the 5% risk level.

Finally, the equallyweighted linear pool of Bayesian predictives tends to be the best

VaR forecaster in a set of 42 forecasting techniques.

KEYWORDS

GARCH models, Bayesian and frequentist estimation, predictivedensity combination, beta linear pool,

censored optimal pooling, backtesting

1INTRODUCTION

Asset returns demonstrate volatility clustering and an abnor-

mal amount of extreme values. The autoregressiveconditional

heteroskedastic (ARCH) model introduced by Engle (1982)

is able to seize these empirical regularities. A more flexible

specification, the generalized ARCH (GARCH) model, was

later proposed by Bollerslev (1986). These models define the

conditional volatility as a deterministic function of past inno-

vations. However, they do not consider the leverage effect,

that is, the asymmetric relation between news and volatility

(Black, 1976). As a consequence, many asymmetric specifi-

cations for conditional volatility appeared around the 1990s

(see, among others, Glosten, Jagannathan, & Runkle, 1993;

Nelson, 1991; Zakoian, 1994). Furthermore, GARCH specifi-

cations were initially coupled with the normal conditional dis-

tribution. However, this appears insufficient to fully account

for the asset return leptokurticity and skewness that can be

empirically observed. Other distributions with fatter tails have

been proposed, such as the standardized Student-tdistribu-

tion (Bollerslev, 1987) or the generalized error distribution

(Nelson, 1991), as well as methods to introduce skew-

ness in these distributions (see, e.g., Fernández & Steel,

1998). Recently, GARCH-type models with complex updat-

ing mechanisms have appeared, such as those obtained from

the generalized autoregressive score modeling framework

(Creal, Koopman, & Lucas, 2013) with skewed and leptokur-

tic conditional distributions.

A predictive density fully depicts the uncertainty related

to a prediction. GARCH-type models can typically be used

to generate predictive densities for future returns of financial

assets (e.g., indexes or stocks). In financial risk management,

precise estimation of the left-tail of asset returns’ predictive

densities is crucial to reliably depict downside risk (Tay &

Wallis, 2000). There are several kinds of predictive densi-

ties possessing different properties. Among them, Bayesian

808 Copyright © 2017 John Wiley & Sons, Ltd. wileyonlinelibrary.com/journal/for Journal of Forecasting.2017;36:808–823.

ARDIA ET AL.809

FIGURE 1 Typical left-tail of predictive densities generated by a

GARCH-type model when we use particular posterior draws (thin solid

lines), when parameter uncertainty is integrated using the Bayesian

approach (bold dashed line), and when we plug ML estimates in the

predictive (bold solid line). The Bayesian predictive is more

conservative than the ML one and accounts for more likely scenarios

[Colour figure can be viewed at wileyonlinelibrary.com]

predictive densities are of particular interest since they

account for parameter uncertainty in a small-sample frame-

work (Geweke & Amisano, 2010). Such predictive densities

can improve out-of-sample left-tail predictive performance

over those that do not integrate parameter uncertainty

(Hoogerheide, Ardia, & Corré, 2012a), and the Bayesian

approach is an appropriate way to account for parameter

uncertainty when the purpose is to produce value-at-risk

(VaR) estimates (Aussenegg & Miazhynskaia, 2006). How-

ever, it has never been shown in the literature that integrat-

ing parameter uncertainty can improve VaR forecasts; we

aim at filling this gap. Figure 1 illustrates why integrat-

ing parameter uncertainty can be useful for left-tail predic-

tion. The Bayesian predictive density (bold dashed line) is

a particular averaging of the predictives that can be formed

with individual posterior draws (thin solid lines). It is gen-

erally more conservative than the predictive density with

plugged maximum likelihood (ML) estimates (bold solid

line) and offers additional flexibility by accounting for all

likely scenarios within the model structure. Nevertheless,

it is also interesting to go beyond this structure by aggre-

gating predictive densities originating from different mod-

els (see, among others, Genest & Zidek, 1986; Gneiting &

Ranjan, 2013; Hall & Mitchell, 2007; Moral-Benito, 2015,

section 5). This extra step allows us to account for model

uncertainty and delivers further flexibility for downside risk

prediction.1

1Some studies already rely on model combination for VaR forecasting

(see Massacci, 2015; Opschoor, van Dijk, & van der Wel, 2015; Pesaran,

Schleicher, & Zaffaroni, 2009). However, they confine themselves to the

linear pool and do not simultaneously account for parameter and model

uncertainty.

In this research, we assess the impact of these two forms of

uncertainty on the left-tail of predictive densities and in par-

ticular on VaR forecasts obtained from these densities. Our

investigations are performed in the universe of GARCH-type

models. Besides having been studied for decades in financial

econometrics, these models are extensively used in the finan-

cial industry. The effect of parameter uncertainty is studied

using Bayesian and ML estimation of GARCH-type models

(Ardia, 2008). The effect of model uncertainty is investigated

using the linear and log pools as well as the recent beta lin-

ear pool. Weights and parameters of the different pools are

computed from past data. Methods for weight computation

include Bayesian model averagingwith predictive likelihoods

(Eklund & Karlsson, 2007), as well as optimal pooling or

OP (Geweke & Amisano, 2011, 2012). Broadly speaking,

the former method averages measures of past predictive per-

formance to form the weights, while the latter looks for the

weights that maximize past predictive performance. We also

use a censoring-based version of OP, referred to as COP, that

allows us to focus on the left-tail (Opschoor et al. 2015).

We contribute to the literature by applying this method to

all of the previously mentioned pools, including the beta lin-

ear pool, and by comparing it to other combination methods,

such as Bayesian model averaging. We investigate whether

COP improves VaR forecasts for combinations of GARCH-

type models.

Large forecasting experiments are carried out with sev-

eral non-nested GARCH-type volatility specifications using

skewed and heavy-tailed conditional distributions. We pre-

dict daily returns of a set of indexes over a window of about

13 years from the early 2000s. For each index, different

predictive densities are produced and aggregated. Then, we

evaluate VaR estimates obtained from individual and com-

bined predictives. We also assess the quality of densities in

the left-tail using probability integral transforms. We find

that Bayesian predictive densities improve VaR estimates at

the 1% risk level for individual models as well as for lin-

ear and log pools. We also find that the VaR backtest is

more robust when linear or log pools are used and that VaR

estimates from these methods are globally better than those

of single models at the 5% risk level. Finally, the equally

weighted linear pool of Bayesian predictives tends to be the

best method for VaR prediction in a set of 42 forecasting

techniques.

The outline of this paper is as follows. Section 2 presents

GARCH-type models. Section 3 describes model estimation

and the different types of predictive densities. Section 4 com-

pares single model predictions in a first application to stock

market indexes. Section 5 discusses the combination of pre-

dictive densities. Section 6 compares single and combined

forecasts in a second application to stock market indexes.

Section 7 concludes.