PORTFOLIO MANAGEMENT: THE ROLE OF CALIBRATION, SHARPNESS, AND UNCERTAINTY

• DOI: http://doi.org/10.1111/jfir.12189
• Date: 01 September 2019
• Author: Shui Ki Wan
• Published date: 01 September 2019

The Journal of Financial Research Vol. XLII, No. 3 Pages 589–608 Fall 2019

DOI: 10.1111/jfir.12189

PORTFOLIO MANAGEMENT: THE ROLE OF CALIBRATION, SHARPNESS,

AND UNCERTAINTY

Shui Ki Wan

Hong Kong Baptist University

Abstract

I evaluate the out‐of‐sample predictability of several major indicators for bull and

bear markets in monthly S&P 500 series with three quadratic probability score

components: calibration, sharpness, and uncertainty. I find that uncertainty limits the

trend characterization and thus provides a new perspective from which to identify

bull and bear markets. I also find that sharpness plays a key role in determining

portfolio returns. Trading strategies that capitalize on sharpness generate higher

Sharpe ratios and portfolio returns. The Aruoba–Diebold–Scotti business conditions

index is the most profitable indicator for both medium‐and long‐term trends.

JEL Classification: C12, C52, C53, G11, G17

I. Introduction

Numerous studies show that stock market returns exhibit different statistical properties

across different market trends, usually termed “bull”and “bear.”For example, the

variance of portfolio returns (Best and Grauer 1991; Dueker 1997; Hamilton and

Susmel 1994; Perez‐Quiros and Timmermann 2000; Turner, Startz, and Nelson 1989)

and the correlation among international equity market returns (Ang and Bekaert 2002;

Longin and Solnik 2001) tend to be higher during bear markets. Therefore, it is equally

important to accurately forecast turning points and formulate trend‐based investment

plans. For example, Guidolin and Timmermann (2005, 2007) examine how

asset allocation depends on market trends and the investment horizon by maximizing

the expected utility of investors. They model excess returns in stocks and bonds,

together with dividend yields, in a multivariate regime‐switching process, and find that

investors allocate more to stocks in bear markets when they have longer investment

horizons. Equally important is to identify indicators with high out‐of‐sample

predictability for bull and bear markets. Chen (2009) and Nyberg (2013) measure

the predictability by quadratic probability score (QPS), an analogue of mean square

prediction error, for each probability forecasting model. They further evaluate the

economic significance by comparing the Sharpe ratios using a simple switching trading

strategy in which the asset allocation depends directly on the position of the probability

forecast relative to an arbitrarily chosen threshold.

This work was supported by the General Research Fund from the Hong Kong Research Grants Council

(Project HKBU No. 255212)

589

© 2019 The Southern Finance Association and the Southwestern Finance Association

In this study, I further investigate the out‐of‐sample predictability of a set

of financial and macroeconomic indicators. I augment QPS with three of its

components: calibration, sharpness, and uncertainty. What motivates this

examination is the recognition that QPS can at best provide a coarse summary

for out‐of‐sample predictive performance. It cannot tell how much squared error

belongs to any nondiversifiable part or the merits and demerits of each forecasting

model. The study by Diebold and Rudebusch (1989) is one of the few that uses the

QPS decomposition proposed by Murphy (1973) to evaluate a set of indicators for

predicting business‐cycle turning points. However, the roles of the QPS

components in portfolio performance and investment strategies are not explored.

The multivariate regime‐switching model suggests an asset allocation decision,

but its objective function depends not only on the estimated smoothing

probabilities for the next market trend, but also on the estimated transition

matrix, the model, and the assumed level of risk aversion. In contrast, the

switching trading strategy is more sensitive to the quality of probability forecasts

surrounding the role to be investigated. Therefore, I use this strategy to evaluate

the economic significance of each of the three components.

Through predicting bull and bear markets in the S&P 500, I argue that the QPS

components address at least three issues. Given the latent nature of stock market trends,

there are a wide variety of methods that can define bull and bear markets. Harding and

Pagan (2003a, 2003b) use simple rule‐based methods to characterize business cycles,

but Kole and van Dijk (2017) find that regime‐switching models that capture both the

variance and mean return outperform rule‐based methods in out‐of‐sample forecasting

and in generating portfolio returns. By decomposing the QPS, I find that the choice of

trend characterization is largely restricted by its level of uncertainty. Intuitively, a

highly uncertain series can severely reduce a model’s predictability and thus lower the

portfolio returns that can be realized. Therefore, the first issue the component analysis

addresses is to directly screen out any trend definitions with unusually high uncertainty

at an early stage.

Second, the component analysis informs the debate as to whether a trading

strategy switching at either the historical average or the symmetric threshold is better

at generating significant economic benefits. Although Nyberg (2013) argues that

switching at the historical average yields higher returns because of its alignment with

the proportion of bear markets (a property of the observed series), I find that the

profitability of a strategy depends more on the sharpness component (a property of

the forecasts). My results reveal that setting the historical average as the threshold is

not always better than the symmetric rule. Models with a higher degree of sharpness

are more profitable when the historical average is used, whereas models lacking

sharpness are more profitable when the symmetric rule is used.

Third, the components provide guidance when formulating a trading strategy.

My simple market‐timing experiment shows that a trading strategy that capitalizes on

the sharpness of the forecasts yields much higher Sharpe ratios and portfolio returns

than a conventional switching strategy. The results are in line with the suggestion in

Gneiting, Balabdaoui, and Raftery (2007) and Gneiting and Raftery (2007) that the

goal of probabilistic forecasting is to maximize the model sharpness subject to

590 The Journal of Financial Research