Forecasting

Definition

Forecasting is the production of probabilistic statements about future values of random variables, given information available up to time $T$ . A complete forecast is a predictive distribution $p(y_{T+h} \mid \mathcal{F}_T)$ ; point forecasts (conditional means or medians), interval forecasts (prediction intervals), and density forecasts are all summaries of this distribution. Forecast quality is evaluated by accuracy (how close predictions are to outcomes) and calibration (whether stated probabilities match empirical frequencies).

Key Ideas

Point forecast: typically the conditional mean $\hat y_{T+h} = E[y_{T+h} \mid \mathcal{F}_T]$ under squared-error loss, or the conditional median under absolute-error loss. Evaluated by Mean Squared Error (MSE) or Mean Absolute Error (MAE).
Interval forecast: a $(1-\alpha)$ -prediction interval $[L_{T+h}, U_{T+h}]$ satisfying $P(y_{T+h} \in [L, U] \mid \mathcal{F}_T) = 1-\alpha$ . Evaluated by coverage rate and interval width.
Density forecast: the full predictive distribution $p(y_{T+h} \mid \mathcal{F}_T)$ . Evaluated by the log predictive score $\log p(y_{T+h} \mid \mathcal{F}_T)$ (strictly proper, cannot be gamed), the CRPS (Continuous Ranked Probability Score), or by probability integral transform (PIT) histograms for calibration.
Forecast horizon: $h$ -step-ahead forecasts require iterating the model $h$ times (iterated forecasts) or estimating a direct $h$ -step regression. Under correct specification both are asymptotically equivalent; under misspecification direct forecasts can dominate (Schorfheide 2005; see Lag-Order Selection Criteria).
Parameter uncertainty: substituting point estimates $\hat\theta$ for unknown parameters understates forecast uncertainty. Bayesian predictive inference integrates over the posterior: $p(y_{T+h} \mid y_{1:T}) = \int p(y_{T+h} \mid \theta) p(\theta \mid y_{1:T}) d\theta$ . See Bayesian Multiperiod Prediction.
Combination: forecast combination — averaging predictions from multiple models — typically outperforms any single model (Bates-Granger 1969). Equal-weight averaging is surprisingly competitive; optimal weights from Forecast Combination (Winkler 1981) use the inverse covariance matrix of forecast errors.

How It Works

Evaluation framework:

Pseudo-out-of-sample evaluation: fix an estimation window ending at $T_0$ ; produce $h$ -step-ahead forecasts from $T_0$ to $T - h$ ; compute accuracy metrics over the evaluation period.
RMSE (Root Mean Squared Error): $\text{RMSE}_h = \sqrt{(T-T_0-h)^{-1}\sum_{t=T_0}^{T-h}(\hat y_{t+h} - y_{t+h})^2}$ .
Diebold-Mariano test: compares MSFEs of two models; test statistic $\bar d / \hat\sigma_d \xrightarrow{d} \mathcal{N}(0,1)$ where $\bar d$ is the mean loss differential.
Theil's U statistic: ratio of model RMSE to random-walk RMSE; $U < 1$ means the model beats the naive benchmark.

Sources of forecast uncertainty (Fair 1980):

(a) Stochastic error terms
(b) + Coefficient uncertainty
(c) + Exogenous variable uncertainty
(d) + Model misspecification

See Forecast Uncertainty Decomposition and Stochastic Simulation.

Why It Matters

Forecasting is the primary empirical test of any economic model — a model that cannot predict out-of-sample carries limited credibility regardless of its in-sample fit. The forecast-combination literature shows that model averaging is typically more reliable than model selection in practice.

Open Questions