RDP 2001-09: What do Sentiment Surveys Measure? 4. What Does Sentiment Measure?
November 2001
- Download the Paper 300KB
While Section 3 has demonstrated that a significant proportion of the variation in the sentiment indices can be explained by lagged information this does not directly address the question of whether sentiment actually measures anything useful, forward looking or otherwise. We take up this question in this section.
The fact that much of the movement in sentiment measures can be explained by other variables does not necessarily mean that respondents are simply backward-looking. Respondents may be forming the best expectations of the future they can and using lagged GDP as a significant input to these expectations. However, if there is no more information in the sentiment surveys than is contained in lagged economic variables then there is little point in poring over the latest release. To assess whether there is any information in the surveys, after allowance is made for the lagged economic variables, we look at regressions of a variety of major economic indicators on our regression residuals.
4.1 Method
Unfortunately, it is difficult to form any expectations about which sentiment series might explain which economic series. While some questions in the sentiment surveys are quite specific, many are vague. For example, the consumer confidence survey asks about ‘general economic conditions’ but it is unlikely that this can be considered to be the same as the quarterly percentage growth rate of real GDP (A). What is more likely is that respondents weight together many economic indicators in forming their response. For this reason we search across all combinations of sentiment and economic variables rather than confining our search on the basis of a priori beliefs.
We choose to focus on GDP, employment, corporate gross operating surplus (GOS), household spending, and retail trade.[16] Initially we conduct simple bivariate regressions (Granger causality tests) of economic variables on sentiment, followed by regressions of economic variables on our residuals. In each regression, an economic variable is regressed on four lags of itself and four lags of a sentiment or residual series. We then compare the predictive power of the residuals to the predictive power of the original sentiment indices, to see if our ‘filtering’ of readily available information from those indices leaves any significant information behind.[17] Results are reported in Tables B6, B7 and B8 (business conditions), and Tables B11 and B12 (consumer confidence) in Appendix B.
In a similar vein, we investigate the extent to which consumer sentiment indicators can be used to predict recessions.[18] We conduct an elementary probit analysis of the average index, its components and the residuals from our earlier regressions, in each case treating the probability of recession as the dependent variable, and a lag of a particular length of a given index or residual series as the independent variable.[19] Results are reported for the first eight lags of each indicator (see Tables B9 and B10 in Appendix B).
Finally, net balance results from business and consumer confidence surveys have an advantage over traditional statistical releases as they are easy to calculate and, as such, are available much sooner than similarly dated economic variables (such as GDP). The Granger causality tests reported above do not directly address the question of whether sentiment indices may be useful as ‘coincident’ indicators, i.e. that they provide an early reading on GDP due to the delays in official statistical releases. To test this we also conducted ‘Granger-causality tests’ where we tested for any significance of contemporaneously dated sentiment variables for contemporaneously dated economic variables.[20] In addition we included contemporaneously dated sentiment in each probit regression. The results did not change as a result of these re-specifications, so we do not report them here. In sum, we find no support for the proposition that sentiment surveys are good coincident indicators.
4.2 Results
An inspection of Tables B5 to B12 suggests that, in general, the residuals from our earlier regressions perform considerably worse than the sentiment indices themselves as predictors for various activity measures and the probability of a recession. This confirms that in most cases the economic information we filtered from the indices explains their predictive success. Nonetheless, the Roy Morgan and Melbourne Institute indices, the NAB Actual Business Conditions index, and some of their components, do continue to predict employment growth after they have been filtered.
To see if the residuals predicted employment growth, controlling for other variables, we added four lags of each residual series to a baseline error-correction model of full-time equivalent employment. We then tested their joint significance; results are reported in Tables B13 and B14.[21] It appears that the filtered Roy Morgan Average Index and Indices 1 and 2 have some predictive power, as does Index 1 from the Westpac-Melbourne Institute survey. This suggests that questions about personal financial conditions may elicit information about the path of employment not provided by ordinary economic indicators. The filtered Actual Business Conditions index is marginally significant, and the Expected Employment indices (3- and 12-month Outlook) also have some predictive power for employment growth. Nonetheless, the economic significance of the indices (as opposed to their statistical significance) is small. Typical results are that a 2 standard deviation change in the sentiment residual (a very large change) leads to a 0.2 per cent change in employment (a relatively small change).
One might have expected that the residuals corresponding to more forward-looking indices would predict the economic variables better than the backward-looking residuals. As noted above, there is evidence that the most forward-looking responses to the NAB survey are less well explained by lagged data than the backward-looking ones. The bivariate regressions suggest, however, that while the three residual series corresponding to the Actual Business Conditions index have some explanatory power, the residuals from the forward-looking indices are without exception redundant. The Expected Employment residuals fare better as predictors of employment growth in an empirical model. But with these exceptions, the unexplained component of the forward-looking indices is simply ‘noise’ rather than being informative about the future.
Consumer sentiment indicators appear to predict recessions up to four or more quarters ahead, but the corresponding residuals are rarely significant, and often not strongly so when they are. The residuals from the second component index in both surveys ‘expected personal financial conditions’) are an exception, however, displaying some predictive power between two and four quarters ahead. As we have seen, the personal financial conditions residuals also help predict employment growth when controlling for other variables.
Footnotes
We also generated results for job vacancies and hours worked, but as these did not differ appreciably from those for employment we do not report them here. For the same reason we do not report results for individual components of household spending (such as spending on food or vehicles). [16]
As the residuals are generated regressors, OLS may not yield correct standard errors (the estimates are, however, consistent). Nonetheless, the analysis of Pagan (1984) suggests that the OLS standard errors are either correct (if no lags are included) or too small (in other cases). As our results suggest that the residuals are generally insignificant, computing corrected standard errors would not change our findings. [17]
Since the business confidence series is quite short (and thus offers only one recession reading) we focus instead on the longer consumer confidence series. [18]
We do not control for other variables. These regressions are conducted in the spirit of obtaining basic descriptive statistics, rather than precise estimates. [19]
We did this by including contemporaneously dated sentiment in the Granger-causality tests in addition to the standard lagged values. [20]
