RDP 2003-01: Business Surveys and Economic Activity 2. The Nature and Content of Business Surveys

Business surveys typically provide qualitative indications from individual companies about a range of questions relating to economic conditions. Firms may be asked to report on how confident they are about the future, as well as on more specific aspects of their business, such as actual and expected sales, profitability, employment and capital expenditure. Quantitative responses (mainly in the form of ranges) are also provided in some surveys, usually relating to areas such as selling prices, input costs and capacity utilisation.

2.1 Advantages

Business surveys potentially provide quite timely information about the current state of the economy, as a number of the survey indicators, such as sales and output, appear to move contemporaneously with economic activity. The survey information has direct parallels in official statistical series but it is released relatively quickly – usually towards the end of the quarter to which the information pertains, or early in the following quarter. In contrast, official data are released with a longer lag because of the large amount of information that has to be collected and collated. National accounts data, for example, are released a little over two months following the end of the quarter.

Survey information can also serve as a cross-check on official statistics, which necessarily tend to be revised as more information becomes available. Moreover, in cases where the survey data are less volatile than the corresponding official data, it may be easier to discern underlying trends from the survey data. This partly accounts for the focus on the Institute of Supply Managers survey in the US (formerly the NAPM survey), the Ifo survey in Germany and the Confederation of British Industry Industrial Trends survey in the UK.^[1]

Survey data can also complement official data by filling in gaps in the official statistics. Examples include series on new orders and capacity utilisation. In some cases, business surveys also permit a different perspective on economic data such as a disaggregation by firm size and/or a distinction between domestically-oriented firms and export-oriented firms. Some surveys also provide supplementary information on specific areas, such as factors impinging on sales and profitability, or from time to time, special questions related to topical issues. Over the past few years, for example, questions have been posed to businesses in Australia relating to the Asian financial crisis and the impact on cash flows of the change in taxation arrangements associated with the introduction of the new tax system.

Expectations about important economic variables may also be important if they lead economic developments in a systematic way. For example, expectations of sales and selling prices are likely to affect investment decisions taken by firms. Thus the business surveys are potentially quite useful for judging the outlook for the economy. Combinations of responses may also be enlightening. For example, divergences in responses between actual and expected business conditions (or confidence), or between actual sales and new orders, may provide clues as to whether a movement in inventories is intended or unintended. Survey-based measures of business inflationary expectations may also be used to derive various measures of short-term real interest rates.

2.2 Disadvantages

Nearly all statistical series are surveys insofar as they are based on samples of firms, households or individuals, rather than a complete enumeration. Thus all surveys, including business surveys, are subject to a number of possible sources of error. Sampling error, which arises as a result of the use of a sample rather than a census, is one source of error, though the likelihood of this type of error can be quantified. Moreover, adjustments can be made to results using standard statistical techniques (for example, responses from a particular industry may be weighted by that industry's share in total output). Other sources of error, however, are more subtle and are difficult to quantify. These relate to issues of survey design such as the ordering of survey questions, the medium of interview, the length of time in a survey and the incentives to respond accurately. These factors can introduce a bias to survey results.

In contrast to most official series, business surveys predominantly provide qualitative, rather than quantitative responses to survey questions, so as to make completing the form easier (and faster) for respondents. For example, respondents usually nominate an increase, decrease or unchanged result for a particular variable (some surveys use up/down or good/poor, while others have gradations within ‘increase’ and ‘decrease’). Even when respondents are asked to provide quantitative responses, this usually takes the form of nominating a given range for a variable, with the collators of the statistics using the midpoint of each range to aggregate responses (for open-ended responses, such as ‘greater than 10 per cent’ for example, the lower bound is generally used).^[2]

Summary statistics are typically used in business surveys to convey the information contained in the qualitative responses. The net balance statistic, which measures the difference between the proportion of firms reporting an improvement in an area and those reporting a deterioration, is the most commonly used statistic.^[3] The net balance statistic allows the presentation of a single figure as a summary of responses to each question. Positive balances tend to be associated with growth in the variable of interest, while negative balances tend to be associated with declines in the variable of interest.

The net balance statistic, however, needs to be used carefully as it is not always clear how firms are reporting their individual experiences. Firms reporting an increase in sales, for example, may be referring to either above-trend growth in sales, or strictly positive growth in sales.^[4] If all respondents are reporting relative to an increasing trend over time, then the numbers reporting ‘increasing’ or ‘decreasing’ should be roughly equal when growth is around trend. However, if some respondents are reporting relative to zero, then when growth is around trend the share reporting ‘increasing’ will be greater than the share reporting ‘decreasing’, suggesting that there may be a positive bias. In order to overcome this type of bias, the simplest adjustment is to refer to the net balance statistic in relation to its long-run average. It should be noted, however, that this adjustment implicitly assumes that the bias does not change over time, whereas in practice it may well do.

Reporting of the net balance statistic can also be confusing. Falls in the level of activity are sometimes inferred from negative net balance statistics, whereas negative net balance statistics can be consistent with positive rates of growth, albeit below trend.

The net balance statistic is also potentially misleading in cases where businesses report gradations of increase or decrease. An upgrading of expectations from ‘unchanged’ to ‘moderate increase’, for example, will result in a rise in the net balance statistic. In contrast, a similar upgrading from ‘moderate increase’ to ‘large increase’ will result in no change in the net balance statistic. Most business surveys employ methods of enumeration to avoid this problem, such as giving different weights to the various categories.

In some cases qualitative results from business surveys are not weighted by the firm's share of output (particularly for results within an industry). This can be misleading if inferences are being drawn about aggregate activity from the survey statistics. Changes in plans of large firms will have a much larger impact on aggregate activity than a similar percentage change of a small firm (this is particularly relevant in industries that are highly concentrated, such as mining and telecommunications).

Business surveys generally provide quantitative ranges for series such as selling prices and capacity utilisation, with respondents being asked to nominate a range for the variable of interest.^[5] These ranges, however, have to be chosen carefully. For example, the ranges determined for nominal variables in a high-inflation environment may be too wide in a low-inflation environment, and therefore less informative. They need to be changed over time (and therefore become less comparable over time). In addition, poor selection of ranges can result in increased volatility in a series owing to respondents switching categories frequently. For example, asking businesses about their expectations for inflation and providing them with reference bands either side of the midpoint of a central bank's target for inflation, can result in considerable volatility in the aggregate series. Volatility may also result from small changes in the distribution of responses (see Kearns (1998)).

Footnotes

For discussion of a number of foreign business surveys see Santero and Westerlund (1996), Cunningham (1997) and Britton, Cutler and Wardlow (1999). [1]

Even in surveys where respondents are asked to nominate single numbers, there is a preference for rounding to numbers such as 0, 5 or 10. This response to uncertainty, and the resultant bunching in the distribution of responses, has important implications for the measurement of central tendency (see Brischetto and de Brouwer (1999)). [2]

A variant on the net balance statistic adds the percentage change answering ‘up’ to one-half of those saying ‘unchanged’. The index is therefore centred around 50, rather than 0. The two approaches are similar in effect, the main difference being the possible range in the net balance statistic and the break-even point. [3]

The degree of imprecision is evident in the large number of respondents who report unchanged conditions. Moreover, the period of comparison is not always clear, with comparisons usually being made with either the previous quarter or the same period a year earlier. [4]

For a discussion of the relative merits of the various types of quantitative responses, see Curtin (2000). [5]