Quantile estimators with orthogonal pinball loss function

• Published date: 01 April 2018
• DOI: http://doi.org/10.1002/for.2510
• Date: 01 April 2018
• Author: Zebin Yang,Ling Tang,Lean Yu

RESEARCH ARTICLE

Quantile estimators with orthogonal pinball loss function

Lean Yu

1

| Zebin Yang

3

| Ling Tang

2

1

School of Economics and Management,

Beijing University of Chemical

Technology, Beijing, China

2

School of Economics and Management,

Beihang University, Beijing, China

3

Department of Statistics and Actuarial

Science, The University of Hong Kong,

Pokfulam Road, Hong Kong

Correspondence

Lean Yu, School of Economics and

Management, Beijing University of Chemical

Technology, Beijing 100029, China.

Email: yulean@amss.ac.cn

Ling Tang, School of Economics and

Management, Beihang University, Beijing

100191, China

Email: tangling_00@126.com

Funding information

Key Program of National Natural Science

Foundation of China, Grant/Award Num-

bers: 71433001 and 71631005; National

Science Fund for Outstanding Young

Scholars, Grant/Award Number:

71622011; National Natural Science

Foundation of China, Grant/Award Num-

ber: 71301006; National Program for Sup-

port of Top‐Notch Young Professionals

Abstract

To guarantee stable quantile estimations even for noisy data, a novel loss

function and novel quantile estimators are developed, by introducing the effec-

tive concept of orthogonal loss considering the noise in both response and

explanatory variables. In particular, the pinball loss used in classical quantile

estimators is improved into novel orthogonal pinball loss (OPL) by replacing

vertical loss by orthogonal loss. Accordingly, linear quantile regression (QR)

and support vector machine quantile regression (SVMQR) can be respectively

extended into novel OPL‐based QR and OPL‐based SVMQR models. The

empirical study on 10 publicly available datasets statistically verifies the

superiority of the two OPL‐based models over their respective original forms

in terms of prediction accuracy and quantile property, especially for extreme

quantiles. Furthermore, the novel OPL‐based SVMQR model with both OPL

and artificial intelligence (AI) outperforms all benchmark models, which can

be used as a promising quantile estimator, especially for noisy data.

KEYWORDS

orthogonal pinballloss function, probability forecasting, quantile estimation, robustness, support

vector machine quantile regression

1|INTRODUCTION

Compared with ordinary point prediction, probabilistic

prediction can generate much more information and has

aroused increasingly attention from both theoretic and

application perspectives (Hyndman & Fan, 2010).

Quantile regression (QR), first proposed by Koenker and

Bassett (1978), can reveal the relationship between the

response and explanatory variables under any quantile,

to provide a probabilistic distribution prediction. For

quantile τ, QR utilizes asymmetric pinball loss to find a

regression curve dividing the dataset properly, where τ

percent points are below the regression curve and 1 −τ

percent points are above. As a result of this virtue, QR

can be applied as univariate and multivariate conditional

quantiles (De Gooijer & Hyndman, 2006), and the QR

method has been applied to a wide range of areas, includ-

ing economics and finance (e.g., Chen, Gerlach, Hwang,

& McAleer, 2012; Covas, Rump, & Zakrajšek, 2014; Rubia

& Sanchis‐Marco, 2013), environment (e.g., Friederichs &

Hense, 2007), physics (e.g., Sohn, Kim, Hwang, & Lee,

2008), etc. However, most real‐world data dynamics can

fall into complex systems, in which various inner factors

interact with each other by following an intricate nonlin-

ear relationship, and the traditional linear QR model

might have difficulty in modeling them.

To address such a drawback of the linear QR model,

various artificial intelligent (AI) models, such as artificial

neural network (ANN) and support vector machine

(SVM), have been introduced to formulate powerful

Received: 23 February 2017 Revised: 2 September 2017 Accepted: 2 December 2017

DOI: 10.1002/for.2510

Journal of Forecasting. 2018;37:401–417. Copyright © 2018 John Wiley & Sons, Ltd.wileyonlinelibrary.com/journal/for 401

AI‐based quantile estimators. Existing studies have fully

proven that the AI‐based quantile estimators, with nonl in-

ear functions and powerful adaptive learning abilities,

significantly outperform the traditional linear QR model

in terms of higher prediction accuracy. For example, by

incorporating the pinball loss function into a three‐layer

feedforward ANN, Taylor (2000) proposed the quantile

regression neural network (QRNN) model. Cannon (2011)

proposed a modified QRNN version by utilizing the Huber

norm to smooth the nondifferentiable loss function. For

SVM, Takeuchi, Le, Sears, and Smola (2006) extended the

SVM for quantile estimation and proposed the SVM

quantile regression (SVMQR), by considering noncrossing

and monotonicity constraints. To reduce computationcom-

plexity, Shi, Huang,T ian, and Suykens (2013) proposedthe

L

1

‐norm‐based SVMQR (L1‐SVMQR) model, in which the

L

2

‐norm regularization used in the SVMQR model is

replaced by a simple but effective L

1

‐norm version.

Furthermore, a high level of noise polluting the real‐

world data might be another tough challenge, and quantile

estimations are often impacted by outliers and noise. In the

vertical loss‐based regression methods (e.g., ordinary least

square regression and pinball loss‐based quantile regres-

sion), the noise of response variables is considered,

whereas the noise in explanatory variables is otherwise

often neglected (Yu & Yao, 2013). Therefore, when pro-

cessing real‐world data, the forecasting results might often

be biased even using powerful AI techniques. Further-

more, this problem may deteriorate in extreme quantile

estimations, since the asymmetric penalty might amplify

the effect of noise. This phenomenon can be referred to

data piling (Hall et al., 2005). Therefore, it is imperative

to establish a more stable quantile estimator, especially

in extreme quantile estimations even for highly noisy data,

which is the main focus of this paper.

Fortunately, orthogonal distance loss may be a

promising alternative to the vertical distance loss, which

can effectively avoid the impact of noise in quantile

regression. Unlike classical vertical loss, which only

considers the noises in response variable, the orthogonal

loss measures the errors both in response and

explanatory variables, which can effectively avoid overly

optimistic estimation. Actually, the orthogonal loss has

already been introduced in ordinary mean regression, to

formulate the total least squares (TLS) model (Golub,

1973; Golub & Van Loan, 1980). Since this seminal work,

various TLS‐based AI models have been developed and

achieved satisfactory forecasting results. For example,

Peng and Wang (2009) introduced the TLS technique into

least square support vector regression (LSSVR) model

and proposed the TLS‐based LSSVR model. Yu and Yao

(2013) incorporated the TLS technique into proximal

SVM (PSVM), and built a new TLS‐based PSVM model

for credit risk evaluation. However, to the best of our

knowledge, there are few studies on quantile regression

using orthogonal loss. Therefore, this paper tends to fill

this gap in the literature by introducing orthogonal loss

to formulate a novel quantile regression loss function

and the corresponding novel quantile estimators, to

address the impact of noise.

Under such a background, this paper aims to propose a

novel quantile regression loss function and the corre-

sponding novel quantile estimators, by introducing the

effective idea of orthogonal loss to guarantee stable

quantile estimation even for noisy data. In particular, a

novel orthogonal pinball loss (OPL) function is first

proposed by utilizing orthogonal loss in the pinball loss

function, instead of vertical loss. Unlike vertical loss,

which only focuses on the noise in response variables,

orthogonal loss measures the loss in both response and

explanatory variables (Yu & Yao, 2013), which can effec-

tively avoid the overly optimistic estimation and thus guar-

antee more stable results even for noisy data. Based on this

new loss function, novel OPL‐based quantile estimators

can be accordingly extended to the classical models of QR

and SVMQR, that is, OPL‐based QR and OPL‐based

SVMQR, respectively. In OPL optimization, a Huber norm

is employed to smooth the nondifferential pinball loss, and

a standard gradient descent algorithm is utilized to gener-

ate final solutions. To test the effectiveness and robustness

of the proposed OPL‐based models, the empirical study

employs 10 publicly available datasets as sample data and

typical quantile forecasting models of QR, QRNN, and

L1‐SVMQR as benchmark models for comparison.

The main motivation of this paper is to formulate a

novel quantile estimation loss function (i.e., the OPL func-

tion) and the corresponding novel OPL‐based quantile

estimators based on the effective tool of orthogonal loss,

and to test their effectiveness and robustness especially

in extreme quantile estimations. The rest of the paper is

organized as follows. Section 2 describes the formulation

process of the novel OPL function and OPL‐based quantile

estimators. The empirical results and further discussions

are presented in Section 3. Section 4 concludes the paper

and outlines further research redirections.

2|METHODOLOGY

FORMULATION

Due to the hidden noise polluting real‐world data, the

phenomenon of overly optimistic estimates (Takeuchi

et al., 2006) might occur in quantile estimations, espe-

cially for extreme quantiles. To guarantee stable quantile

estimations even for noisy data, this paper aims to

introduce orthogonal distance for measuring quantile

402 YU ET AL.