Natural Gas Storage Forecasts: Is the Crowd Wiser?

• Author: Fernandez-Perez, Adrian

1. INTRODUCTION

   Inventory information is of critical importance in natural gas price discovery because it measures the supply and demand fundamentals directly. In line with the theory of storage of Kaldor (1939), Working (1949), and Brennan (1958), an increase (decrease) in demand or a decrease (increase) in supply is likely to reduce (increase) inventories and hence increase (decrease) the expected price of natural gas. In recognition of its importance, a set of mechanisms to facilitate the flow of information on gas storage has emerged. The weekly U.S. Energy Information Administration's (EIA) Natural Gas Storage report measures the change in the number of cubic feet of natural gas held in underground storage during the past week (source: http://ir.eia.gov/ngs/ngs.html). The reports are widely followed by market participants, and their releases often trigger large price movements (e.g., Linn and Zhu, 2004; Gay et al., 2009; Halova et al., 2014). (1)

   Professional analysts play a key role in helping market stakeholders to predict EIA's natural gas storage announcements. Their consensus forecast is often regarded as a benchmark against which the information-content of the EIA's announcement can be assessed (e.g., Ye and Karali, 2016; Rousse and Sevi, 2019). For instance, Gu and Kurov (2018) show that the difference between the median forecast of analysts with high historical forecast accuracy and the consensus forecast can be used by traders to predict inventory surprises. Professional analysts' forecasts also facilitate price discovery in the futures markets by carrying additional information beyond seasonal patterns and past storage flows (e.g., Gay et al., 2009; Ederington et al., 2019). While the contributions of professional forecasts are well-established, the benefits of crowdsourced forecasts for natural gas stakeholders remain unknown.

   A recent development in the market for analyst forecasts is the emergence of crowdsourced forecasts that intend to compete with those of professionals. Crowdsourcing is defined as a task normally performed by professionals but outsourced to a large network of people via an open call (Jame et al., 2016). The key ingredients of crowdsourcing are an organization that has a task it needs performed, a community that is willing to perform the task, an online environment that allows the work to take place and the community to interact with the organization, and a mutual benefit for the organization and the community (Brabham, 2013). Websites such as Estimize aggregate forecasts of individuals on numerous events (e.g., firm earnings announcements, target federal funds rates, or consumer price indices). Estimize is an "open financial estimates platform designed to collect forward-looking financial estimates from independent, buy-side, and sell-side analysts, along with those of private investors and academics" (source: www.estimize.com/about). As of today, the platform has provided forecasts for over 80 economic indicators and numerous earnings estimates across developed and major emerging markets. More than 88,000 contributors have provided a total of more than 2.7 million economics and earnings estimates.

   While Estimize has been providing crowdsourced forecasts for the EIA's natural gas storage announcements since the first quarter of 2014, it is unclear whether natural gas market stakeholders can benefit from considering such information on top of forecasts provided by professional analysts (e.g., Reuters, Bloomberg, etc.). On the one hand, crowdsourced forecasts may be more accurate and informative, thus improving price discovery and helping market stakeholders form better predictions about the EIA's natural gas storage changes. Greater accuracy of crowdsourced forecasts may arise from the lower exposure of crowd analysts to behavioral and rational biases affecting professional analysts because of reputation or career concerns. For instance, Harford et al. (2018) show that equity analysts strategically allocate more effort to firms that are relatively more important for their careers (see also Ramnath et al. (2008) for a survey on sell-side analysts' biases). The ability to provide new information may also arise from a wider diversity of the crowd analysts. In that sense, Adebambo et al. (2016) show that crowdsourced analysts tend to have more varied backgrounds than professional analysts. A more diverse collection of people can bring new information to the market when the diversity of opinions (assumptions, forecast models, data) allows for a broader information set compared to those provided by professional analysts (Surowiecki, 2005; Lang et al., 2016). Finally, one of the key incentives of crowd forecasters is social-recognition (CrowdWorx, 2012) (2). Each forecaster has a private interest to produce the forecast that outperforms all competing forecasts to gain publicity. Hence, crowd forecasters should collect and process relevant information to produce the most accurate forecast possible (see Laster et al. (1999)).

   On the other hand, there are also reasons why crowdsourced forecasts may be less accurate and informative than professional ones. First, professional analysts have access to greater resources, have greater monetary incentives, and more professional experience, leading to greater forecast accuracy (e.g., Clement, 1999). Accordingly, the forecasts of crowd analysts may not improve the accuracy of the average estimate of better incentivized and trained professionals. Moreover, diversity is a double-edged sword (Milliken and Martins, 1996). The greater diversity within the group of crowd analysts can increase the costs of gathering information (e.g., Giannetti and Yafeh, 2012). If the divergence of opinions and forecasting methods is too pronounced, analysts may underweight or even ignore information produced by other analysts, resulting in less informative consensus forecasts.

   Given the mixed theoretical arguments, the usefulness of crowdsourced relative to professional forecasts for natural gas storage changes is an empirical question that we examine in this paper. Our sources of professional and crowdsourced forecasts are the net change in gas storage figures from Thomson Reuters Economic Polls and Estimize, respectively. (3) We analyze 231 weekly EIA's natural gas storage announcements over the 2014-2018 period encompassing 5,112 and 2,458 Reuters and Estimize forecasts, respectively. We find that, on average, crowdsourced forecasts are less accurate than those of professionals. This difference is significant, both statistically and economically. Over the sample period, Reuters forecasts have an average absolute percentage error of 22.5% from the actual value of the natural gas storage change announced by the EIA. In contrast, Estimize forecasts have an average absolute percentage error of 48.2%. There is, therefore, a substantial difference of 25.7% between the forecast accuracy of Reuters and Estimize. These results suggest that crowd analysts do not utilize relevant information that would be missed by professionals, nor do they process existing information in a way that leads to greater accuracy.

   We next examine possible reasons why, on average, crowdsourced forecasts are less accurate than their professional counterparts. We observe a greater divergence of opinions among crowd analysts. This finding is consistent with crowd analysts sharing less common beliefs due to their greater diversity (e.g., Lang and Lundholm, 1996). (4) In other words, when the divergence of opinions is pronounced, the aggregate forecast tends to deviate more from individual forecasts on average and becomes less likely to reflect the set of information of each analyst. This is the first plausible explanation for the lower accuracy of the crowdsourced forecasts. We further document evidence suggesting lower incorporation of publicly available information in crowdsourced relative to professional forecasts. This finding is consistent with Estimize forecasters having their own views and being less affected by monetary incentives and career concerns that push professional analysts to herd more on the publicly available information (e.g., Scharfstein and Stein, 1990; Welch, 2000). The finding can also be explained by a greater need for social-recognition that leads crowd analysts to issue more extreme forecasts (i.e., to rely on an even narrower set of private information) in order to differentiate themselves from others (Laster et al., 1999). The greater reliance on private, as opposed to public information of crowd analysts, may in turn explain the wider disagreement we observe. More importantly, it may contribute to lower accuracy of the crowdsourced forecasts on average because of the lower incorporation of publicly available information on natural gas storage already contained in professional analyst forecasts. Consistent with this interpretation, Pierdzioch and Rulke (2012) find that forecaster anti-herding (overweighting private over public information) is negatively correlated with forecast accuracy. Using multivariate regression, we test more directly whether these two differences (wider disagreement and lower incorporation of publicly available information) help explain the difference in accuracy between professional and crowdsourced forecasts and find supportive results.

   We further investigate market reactions (price, volume, and volatility) to the EIA announcements. More specifically, we are interested in information the market infers from Reuters and Estimize consensus forecasts. Based on prior literature, we expect the market to anchor its expectation of gas storage change in the analyst consensus forecast and to react to a marked difference between the announced actuals and market expectations (reflecting the unexpected news content or surprise of the EIA release). We find that market reacts to a departure of the announced actuals from the Reuters consensus...