In this issue of the Journal of Economic Issues, Dr. Taseen has taken The Wall Street Journal survey forecast data originally analyzed by Mark Greer (1999) and extended Greer's analysis to the issue of the prediction of turning points in macroeconomic variables. (1) Greer found that the consensus (arithmetic mean) forecast of The Wall Street Journal's panel of professional forecasters performed no better than coin flipping at predicting the direction of change in macroeconomic variables. By contrast, Taseen, using the same data, concludes that the mean forecast performed much better than would be expected by chance at predicting turning points in the variables. Taseen draws this conclusion because, in well over 50 percent of the cases where a turning point occurred, the consensus forecast predicted that one would occur. (2)
Taseen is correct that 75 percent of the turning points were anticipated by the consensus forecast; however, his conclusion that the consensus forecast performed appreciably better than coin flipping at predicting turning points does not follow. The basic flaw in Taseen's analysis is that he only considers cases where a turning point occurred. He ignores years where a turning point did not take place and excludes from his analysis instances where the consensus forecast erroneously predicted that a turning point would occur. There were many such false calls in the period examined.
In order to understand the shortcoming with Taseen's methodology, consider a forecasting technique that amounted to the simple rule of always forecasting a turning point. Such a forecasting technique would achieve a 100 percent success rate under Taseen's forecast evaluation methodology, since it would predict every actual turning point. However, one would not want to not want to conclude that such a forecasting rule would perform better than coin flipping at predicting whether a turning point will occur or that the rule would have any value to a forecast user. In order to assess the efficacy of a group of forecasters at forecasting turning points, one also needs to know how they performed when no turning point occurred.
Tables 1-5, using the same data as Greer 1999, report the data pertaining to actual and predicted turning points in their entirety, including cases where there was not a turning point. Table 6, a contingency table, compiles the predicted and actual outcomes for all the variables.
As one can see, when one takes into consideration...