Forecasting with many predictors using Bayesian additive regression trees

• Date: 01 November 2019
• Author: Jan Prüser
• DOI: http://doi.org/10.1002/for.2587
• Published date: 01 November 2019

Received: 17 September 2018 Revised: 26 February 2019 Accepted: 8 March 2019

DOI: 10.1002/for.2587

RESEARCH ARTICLE

Forecasting with many predictors using Bayesian additive

regression trees

Jan Prüser1,2

1Faculty of Economics and Business

Administration, University of

Duisburg-Essen, Essen, Germany

2Ruhr Graduate School in Economics,

RWI—Leibniz Institute for Economic

Research, Essen, Germany

Correspondence

Jan Prüser, RuhrGraduate School in

Economics, RWI—Leibniz Institute for

Economic Research, Hohenzollernstrasse

1–3, D-45128 Essen, Germany.

Email: jan.prueser@rgs-econ.de

Abstract

Forecasting with many predictors provides the opportunity to exploit a much

richer base of information. However, macroeconomic time series are typically

rather short, raising problems for conventional econometric models. This paper

explores the use of Bayesian additive regression trees (Bart) from the machine

learning literature to forecast macroeconomic time series in a predictor-rich

environment. The interest lies in forecasting nine key macroeconomic vari-

ables of interest for government budget planning, central bank policy making

and business decisions. It turns out that Bart is a valuable addition to existing

methods for handling high dimensional data sets in a macroeconomic context.

KEYWORDS

fat data, forecasting, nonlinearity, variable selection

1INTRODUCTION

High-dimensional inference and machine learning meth-

ods currently gain in popularity in many fields. In macroe-

conomic applications they have the potential to utilize

a wide range of predictors. Government statistical agen-

cies collect data on a wide range of macroeconomic

variables—for example, measures of output, capacity,

employment and unemployment, prices, wages, housing,

inventories, orders, stock prices, interest rates, exchange

rates, and monetary aggregates. Forecasting with many

predictors provides the opportunity to exploit a much

richer base of information. However,macroeconomic time

series are typically rather short. A large amount of predic-

tors combined with only a small number of observations

raises problems for conventional econometric methods.

Intuitively, there is not enough information in the data

to estimate large models in an unrestricted fashion. So

far, the factor model estimated with principal components

(see Stock & Watson, 2002) and shrinkage methods like

the Lasso estimator (see Korobilis, 2013b; Stock & Wat-

son, 2012) have been found to be useful in addressingthese

problems. The factor model saves degrees of freedom by

summarizing the data through a few factors which are

then used as predictors. The Lasso shrinks the coefficients

towards zero in a data-driven way in order to avoid over-

fitting. This paper explores the use of Bayesian additive

regression trees (Bart), proposed by Chipman, George, and

McCulloch (2010), for forecasting with many predictors in

a macroeconomic context.

The Bart model is a sum-of-tree model and is attrac-

tive for several reasons. In order to address the problem

of a short span of observations relative to the number of

potentially relevant explanatory variables, the Bart model

provides built-in variable selection. Furthermore, most

econometric models are linear. In contrast,the Bart model

allows for nonlinearity and interaction effects between

predictors in a natural way. However, it is well knownthat

large tree models tend to overfit (i.e., they fit the noise

in the data rather than detecting a pattern which is use-

ful for forecasting). The Bayesian approach provides the

opportunity to address this problem by using prior distri-

butions to regularize the fit of each individual tree, so that

each tree only explains a small fraction of the variation in

the response variable. In addition, the Bayesian approach

allows one to account for parameter, tree, and forecasting

Journal of Forecasting. 2019;38:621–631. wileyonlinelibrary.com/journal/for © 2019 John Wiley & Sons, Ltd. 621