Booth Hyndman Tickle and De Jong 2006 — Lee-Carter Mortality Forecasting A Multi-Country Comparison of Variants and Extensions

Summary

A 10-country out-of-sample evaluation of five Lee-Carter variants and extensions — LC (original), LM (Lee-Miller), BMS (Booth-Maindonald-Smith), HU (Hyndman-Ullah), and DJT (De Jong-Tickle) — using data fitted to 1985 and evaluated on the 1986–2000 period. The headline finding is twofold: all four non-original methods substantially beat LC on log death rate accuracy (by up to $61\%$), but there are no statistically significant differences among the five methods in life expectancy (LE) forecast accuracy ($p = 0.21$). The paper also performs a factorial decomposition of the three factors defining the LC variants (fitting period, adjustment method, jump-off rates), identifying fitting period length and jump-off bias as the dominant drivers of LC's errors. Published as Working Papers in Demography No. 101, ANU; also published as Booth et al. (2006), Demographic Research 15(9): 289–310.

Key Claims

• All four variants beat LC on log death rates: Mean absolute error (MAE) in log death rates is approximately halved (male) to doubled (female) relative to LC. The improvements range from $39\%$ to $61\%$ across the 10 countries. However, after Tukey's Honest Significant Differences correction, the original LC is the only method significantly different from the others; LM, BMS, HU, and DJT are not significantly different from each other (all pairwise $p > 0.86$).
• No significant differences for life expectancy: A 2-way analysis of variance (ANOVA) on mean absolute LE errors finds no significant difference between the five methods ($p = 0.21$). BMS provides the numerically best LE forecasts (male $0.69$, female $0.38$) but this advantage is statistically insignificant.
• Quantitative summary:
  • Log death rate MAE (male/female average): LC $= 0.31/0.45$; LM $= 0.17/0.17$; BMS $= 0.15/0.16$; HU $= 0.15/0.15$; DJT $= 0.15/0.15$
  • Life expectancy MAE: LC $= 0.89/0.66$; LM $= 1.05/0.43$; BMS $= 0.69/0.38$; HU $= 0.78/0.54$; DJT $= 0.98/0.44$
• Factorial decomposition of LC variants ($3 \times 4 \times 2 = 24$ combinations of fitting period $\times$ adjustment $\times$ jump-off):
  • Fitting period is the dominant factor: short fitting periods (LM/BMS) consistently outperform the long LC period on log death rates.
  • Jump-off rates: actual jump-off rates generally outperform fitted rates, especially when the fitting period is long; advantage diminishes with short fitting periods. Jump-off error is particularly large for the original LC because it combines a long fitting period with fitted rates.
  • Adjustment method: marginal effect relative to the other two factors. Adjustment to $D_t$ (total deaths, original LC method) consistently produces the largest errors. When fitted jump-off rates are used, any adjustment worsens log death rate forecasts because it renders the model statistically non-optimal.
  • Best combination for log death rates: short fitting period (BMS) + no adjustment + fitted rates → MAE $= 0.154$, a $60\%$ improvement over LC (MAE $= 0.384$).
  • Best combination for life expectancy: short fitting period + actual rates → MAE $= 0.484$ ($38\%$ improvement over LC's $0.775$).
• Conceptual contribution: Accuracy in log death rates does not translate into accuracy in life expectancy. The composite transformation (exponentiation + life table computation, with implicit age-weighting and complex error cancellation) means that LE forecast accuracy is "largely a matter of luck." Evaluation of forecast error in the underlying log death rates is essential; LE accuracy alone is insufficient and can be misleading about the relative quality of methods.
• LM as de facto standard: The LM variant is widely referred to as "the Lee-Carter method" and is now standard practice. However, this evaluation shows that LM is not the optimal variant even within the LC family — the simple modification of using a short fitting period with no adjustment and actual jump-off rates outperforms LM for log death rates.
• HU and DJT advantages beyond accuracy: Both methods produce age-specific forecasts that are smooth across age (via functional smoothing and B-spline constraints respectively), which is advantageous for applications requiring realistic age profiles. Both also provide natural extensions for multi-population forecasting. These qualitative advantages are not captured by the MAE metrics.

Concepts Introduced or Extended

• Lee-Carter Model

Entities Mentioned

• Heather Booth
• Rob J. Hyndman
• Leonie Tickle
• Piet de Jong

My Take

The headline result — no significant difference among the four non-original methods for life expectancy — substantially qualifies the complexity argument for HU and DJT. The sophisticated nonparametric and state-space machinery of HU and DJT achieves essentially the same LE accuracy as the simple LM fix (use 1950 as start year, jump off from actual rates). The paper's strongest contribution may be methodological rather than empirical: the demonstration that LE accuracy is a poor criterion for discriminating among forecasting methods. The factorial decomposition is also valuable — it shows that the improvements attributed to "better methods" are largely attributable to two simple choices (shorter fitting period and actual jump-off rates) that any practitioner can apply without adopting the full HU or DJT machinery. The practical recommendation is therefore: if you want better log death rates, use a short fitting period and actual jump-off rates, with no adjustment of $k(t)$. If you care primarily about life expectancy, almost any modern variant will do about equally well.