RDP 2008-02: Combining Multivariate Density Forecasts Using Predictive Criteria 1. Introduction

Density forecasts, or fan charts, can help to communicate risks around a central tendency, or point forecast. Density forecasts are useful tools for inflation-targeting central banks as they can be used to quantify the probabilities of key variables being outside a given range in the future. Furthermore, multivariate or joint density forecasts can be useful in predicting the covariances across different variables of interest.

Under a Bayesian estimation framework, constructing density forecasts using a statistical model is straightforward, enabling the various types of uncertainty inherent in forecasts to be incorporated in a coherent fashion. Taking multiple draws from a model's posterior parameter distribution allows for parameter uncertainty in the forecasts. Taking many draws from a model's assumed distributions for shocks can help to characterise an inherently uncertain future. But using a single model may not result in an accurate characterisation of the true degree of uncertainty since the true data-generating process is unknown. Forecast uncertainty due to model uncertainty can also be considered by combining several models.

There is considerable evidence that combining point forecasts from multiple models can improve forecast accuracy (see Timmermann 2006). Much less attention has been paid to combining density forecasts. Some recent work filling this gap includes Kapetanios, Labhard and Price (2005), Hall and Mitchell (2004, 2007) and Jore et al (2007). While point forecast combinations are usually evaluated according to root mean squared errors (RMSE), the criteria for evaluating density forecasts are less clear cut. This is primarily because the true density is never observed. Unlike ‘optimally’ combined point forecasts, there is nothing to guarantee that a combined density forecast will perform better even in-sample, let alone out-of-sample. Also, as Hall and Mitchell (2004, 2007) note, a combined density may have characteristics quite different to those of the individual densities from which it is constructed. For example, the weighted linear combination of two normal densities with different means and variances will be non-normal. So while density forecasts from a combination of models are more flexible than a density constructed from a single model, whether or not the combined density provides a more accurate description of the true degree of uncertainty is, in the end, an empirical question and will depend on the method used to choose weights for individual models when constructing the combined density.

Most of the previous literature on combining density forecasts has focused on univariate densities, that is, density forecasts for a single variable. Yet in many settings it is of interest to characterise the joint probabilities of future outcomes of several variables. For instance, a policy-maker might be interested in the joint probabilities of a target variable and a policy instrument.

This paper proposes to combine multivariate density forecasts from a suite of models consisting of a Bayesian vector autoregression (BVAR), a factor-augmented vector autoregression (FAVAR) and a dynamic stochastic general equilibrium (DSGE) model. A weighting scheme based on predictive likelihoods following Eklund and Karlsson (2007) and Andersson and Karlsson (2007) is used to combine the models. This weighting scheme also allows for different weights to be assigned to different models at different forecast horizons. We evaluate the combination forecasts following Diebold, Gunther and Tay (1998) and Diebold, Hahn and Tay (1999) by assessing whether the probability integral transform of a series of observations with respect to the density forecasts are uniformly distributed and, in the case of the one-step-ahead forecasts, also independently and identically distributed.

The rest of the paper is structured as follows. Section 2 outlines the suite of models and describes how they are estimated. Section 3 presents some density forecasts and discusses the motivation for using an out-of-sample-based weighting criteria to combine the models. The predictive-likelihood weighting scheme is outlined here. Section 3 also discusses the trade-offs implied by choosing the lengths of the training and hold-out samples necessary to evaluate an out-of-sample predictive criteria. Section 4 describes the data and the model weights obtained. A univariate and multivariate evaluation of the combined density forecasts is presented in Section 5. The final section concludes.