Forecast Uncertainty Decomposition

Definition

Forecast uncertainty decomposition is a stochastic-simulation method that partitions the total expected squared forecast error of an econometric model into four hierarchically nested components — randomness in error terms, uncertainty in coefficient estimates, uncertainty in exogenous-variable forecasts, and model misspecification — so that the relative contribution of each source can be measured and reported alongside point forecasts.

Key Ideas

• The four components are strictly ordered by scope: $\sigma^2_a \leq \sigma^2_b \leq \sigma^2_c \leq \tilde{\sigma}^2(x)$; each increment reveals the marginal cost of one additional source of uncertainty.
• Component (a) — stochastic error terms only: draw error terms from estimated distributions, hold coefficients fixed at point estimates, treat exogenous variables as known.
• Component (b) — adds coefficient uncertainty: also draw coefficient vectors from their estimated joint distribution.
• Component (c) — adds exogenous-variable uncertainty: also draw exogenous-variable forecasts from their own distributions or models.
• Component (d) — misspecification: the gap between the outside-sample squared residual and the stochastic-simulation variance, $d_i(t,k,\ldots) = \hat{e}^2_i - \tilde{\sigma}^2_i$.
• Total variance is computed in closed form under the constancy assumption: either $\tilde{\sigma}^2(x) + \bar{d}_i(k)$ [no-trend, eq. 4] or $\tilde{\sigma}^2(x) + \bar{d}_i(k) \cdot \bar{y}^2_i$ [trend, eq. 5].
• The method is prospective: it does not require waiting for the forecast period to conclude, unlike root-mean-square error (RMSE)-based evaluation.

How It Works

1. Estimate the model on the in-sample period (e.g., 1954I–1977IV).
2. Stochastic simulation: run $J$ trials (Fair uses 1000–2000), each drawing error terms (and optionally coefficients, exogenous variables) from their estimated distributions; compute the simulation variance $\tilde{\sigma}^2_i$ for each endogenous variable $i$ and horizon $k$.
3. Rolling reestimation for misspecification: re-estimate the model on each of $T$ rolling windows terminating one period before the forecast origin, solve the model one step ahead, and compute the outside-sample residual $\hat{e}^2_i(t) - \tilde{\sigma}^2_i(t)$ for each window. Average over $T$ to obtain $\bar{d}_i(k)$.
4. Add the components according to eq. 4 or 5 to obtain total expected squared error.
5. Compare models by their total variances across horizons and variables.

Why It Matters

• Distinguishes structural model quality (misspecification) from sampling uncertainty (coefficients) from input uncertainty (exogenous variables) — information that RMSE collapses into a single number.
• Practically important finding: exogenous-variable uncertainty often dominates in large macroeconometric models, redirecting attention from in-sample fit to exogenous-input forecasting.
• Provides prospective confidence intervals that forecasters can report before outcomes are observed, improving communication of uncertainty.
• Anticipated the forecast-comparison and model-evaluation literature (Diebold-Mariano, forecast combination) by providing a principled benchmark framework.

Open Questions

• Constancy assumption: Fair acknowledges that assuming $d_i(t,k)$ has a constant mean over time is strong and untestable within the model. Structural breaks or improving/deteriorating model quality violate it.
• Dimensionality: For models with many endogenous variables, the misspecification rolling-reestimation step is computationally expensive; approximation shortcuts are an open design question.
• Non-linear models: The derivation assumes linearity (or near-linearity) for variance aggregation; Dynamic Stochastic General Equilibrium (DSGE) and nonlinear macro models require simulation-based generalizations.
• Covariance across variables: The method as stated focuses on marginal variances; joint uncertainty (e.g., for policy scenarios) requires extending to covariance matrices.

Related

• Stochastic Simulation
• Macroeconometric Model
• Misspecification
• Model Comparison
• Forecasting
• Ray C. Fair