Probability Integral Transform

Definition

The probability integral transform (PIT) evaluates a sequence of density forecasts by mapping each realized outcome through its own predictive CDF: $u_t = F_t(y_t)$, where $F_t$ is the forecast distribution made for period $t$. If the density forecasts are correct, the PIT values $\{u_t\}$ are i.i.d. Uniform$(0,1)$ — so checking uniformity and independence of the PITs tests the absolute adequacy of a predictive distribution (Rosenblatt 1952; Diebold-Gunther-Tay 1998). Geweke-Amisano (2010) use it as the frequentist evaluation counterpart to Bayesian predictive-likelihood comparison (Geweke-Amisano 2010).

Key Ideas

• The transform. For continuous $y_t$ with predictive CDF $F_t$, $u_t=F_t(y_t)$; correct forecasts $\Rightarrow u_t\sim$ Uniform$(0,1)$, i.i.d. Non-uniformity diagnoses miscalibration; serial dependence in the $\{u_t\}$ diagnoses missed dynamics.
• Reading the PIT histogram. A hump-shaped histogram means the forecasts are too wide (overdispersed); a U-shape means too narrow (overconfident / tails too thin); asymmetry means location/skew bias. This is the density-forecast analogue of a calibration reliability diagram.
• Comparison vs. evaluation — two distinct questions. Comparison asks which of several models predicts best relative to each other; evaluation asks whether a given model is adequate in absolute terms. The PIT answers evaluation; predictive likelihoods answer comparison.
• Bayesian predictive likelihood (the comparison side). Bayesian inference yields exact out-of-sample predictive densities that coherently integrate over parameter uncertainty; the predictive likelihood / log predictive score (whose cumulative differences are log predictive Bayes factors) ranks models — an inherently Bayesian, strictly proper criterion (see Proper Scoring Rule, Model Comparison).
• Complementarity (Geweke-Amisano). Applied to five models of daily S&P 500 returns (1976–2005) — Gaussian, GARCH, $t$-GARCH, stochastic volatility, hierarchical Markov normal mixture — the Bayesian comparison and the frequentist PIT evaluation are complementary: each reveals model strengths and weaknesses the other misses.

Why It Matters

• Absolute adequacy, not just ranking. A model can win a comparison yet still be badly calibrated; the PIT catches that, guarding against declaring a "best" model that is nonetheless wrong.
• Standard density-forecast diagnostic. PIT histograms and their transforms (e.g. inverse-normal for autocorrelation tests) are the workhorse check for interval and density forecasts across macro and finance.
• Bridges Bayesian and frequentist evaluation. It pairs a coherent Bayesian comparison metric with an assumption-light frequentist adequacy test — using both is more informative than either alone.

Open Questions

• Estimation effect and dependence. The i.i.d.-Uniform null ignores parameter-estimation uncertainty and can be distorted by serial dependence; adjusted tests are needed.
• Multi-step and multivariate PITs (Rosenblatt chains) are harder to interpret and test.
• Power. PIT-based tests can have low power against some misspecifications; combining with scoring rules helps.

Related

• Proper Scoring Rule
• Model Comparison
• Probability Calibration
• Forecasting
• Volatility Forecast Evaluation
• Geweke-Amisano (2010) Comparing and Evaluating Bayesian Predictive Distributions of Asset Returns
• John Geweke
• Gianni Amisano