Christoffersen (1998) Evaluating Interval Forecasts

Summary

This paper builds the standard framework for evaluating interval forecasts (prediction intervals) under conditional heteroskedasticity, and is the foundation of modern value-at-risk backtesting. Christoffersen observes that the existing literature tests only for correct unconditional coverage — whether the interval contains the outcome the right fraction of the time on average — which is inadequate when volatility is time-varying, because a forecaster can hit the right average coverage while getting the timing of violations badly wrong (clustered violations in turbulent periods). He formalizes conditional coverage through the "hit" indicator sequence and derives simple likelihood-ratio tests, illustrated on exchange-rate risk-management interval forecasts.

Key Claims

• The hit indicator. For a sequence of $(1-p)$ interval forecasts, define $I_t=1$ if the realization falls inside the interval (equivalently, for a VaR, if the loss does not exceed the quantile) and $I_t=0$ otherwise. Everything is evaluated through this binary sequence.
• Correct conditional coverage = i.i.d. Bernoulli. An interval forecast has correct conditional coverage if and only if $\{I_t\}$ is independent and identically distributed Bernoulli($p$): the violation rate is right and violations are not predictable from the past.
• Unconditional coverage test ($LR_{uc}$). A likelihood-ratio test of $\mathbb E[I_t]=p$ (the empirical violation frequency equals the nominal rate) — the only property the earlier literature checked.
• Independence test ($LR_{ind}$). Model $\{I_t\}$ as a first-order Markov chain and test that the probability of a violation does not depend on whether yesterday was a violation ($\pi_{01}=\pi_{11}$); rejects clustering of violations even when average coverage is correct.
• Conditional coverage test ($LR_{cc}$). The joint test decomposes exactly as $LR_{cc}=LR_{uc}+LR_{ind}$ (asymptotically $\chi^2_2$), so a forecaster can be diagnosed as failing on coverage, on independence, or both.
• Why it matters for risk management. Under GARCH-type dynamics, a VaR model that ignores time-varying volatility will suffer violation clusters in volatile periods; only a conditional-coverage evaluation detects this, which is why $LR_{cc}$ became the standard VaR backtest.

Concepts Introduced or Extended

• Interval Forecast Evaluation
• Value at Risk
• Volatility Forecast Evaluation

Entities Mentioned

• Peter F. Christoffersen

Quotes

"Most of the literature implicitly assumes homoskedastic errors even when this is clearly violated, and proceeds by merely testing for correct unconditional coverage. Consequently, I set out to build a consistent framework for conditional interval forecast evaluation."

My Take

The elegance is in the reduction: no matter how complicated the forecasting model, its interval forecasts are judged entirely through a sequence of 0/1 hits, and "a good interval forecast" becomes the crisp, testable statement "the hits are i.i.d. Bernoulli($p$)." Splitting that into a coverage part and an independence part is what makes the test diagnostic rather than merely pass/fail — it tells you how the model is wrong. For the wiki this is the evaluation counterpart to the VaR estimation machinery: the HS-VaR pathologies documented there (right average coverage, disastrous conditional coverage after a volatility spike) are exactly what $LR_{ind}$ is built to catch. The main limitation, addressed by later work (Engle-Manganelli's DQ test, Berkowitz density tests), is that the first-order-Markov independence alternative has low power against richer forms of violation dependence.