P-spline Mortality Smoothing

Definition

P-spline mortality smoothing models the two-dimensional surface of log-mortality rates $\log\mu_{xt}$ — indexed by age $x$ and calendar year $t$ — as a tensor product of B-splines with difference penalties, estimated within a Poisson generalized linear model (GLM). Deaths $d_{xt} \sim \text{Poisson}(e_{xt}\mu_{xt})$ where $e_{xt}$ is exposure. The log-mortality surface is:

$$\log\mu_{xt} = \mathbf{B}_x \boldsymbol{\Theta} \mathbf{B}_t^\top$$

where $\mathbf{B}_x$ and $\mathbf{B}_t$ are B-spline bases for age and year, and the coefficient matrix $\boldsymbol{\Theta}$ is penalized by $\lambda_x P_x + \lambda_t P_t$ — second-order difference penalties in the age and year directions respectively (Currie, Durbán, and Eilers 2004).

Key Ideas

• P-splines: Penalized B-splines (Eilers and Marx 1996) combine B-spline flexibility with a difference penalty on adjacent coefficients that prevents overfitting and encourages smooth extrapolation approximately polynomial beyond the data range.
• Two-dimensional surface: Rather than modeling age and time separately (as in Lee-Carter's bilinear $a_x + b_x k_t$), P-splines fit the mortality surface as a whole, allowing any smooth age-year interaction pattern.
• GLAM estimation: The Generalized Linear Array Model (GLAM) exploits the Kronecker product structure of the tensor-product basis to estimate via iteratively reweighted least squares (IRLS) on coefficient arrays, avoiding construction of the full vectorized design matrix — critical for computational feasibility.
• Unified smoothing and forecasting: The penalty in the year direction governs both in-sample smoothness and out-of-sample extrapolation; no separate time-series model for a latent index is needed.
• Cohort effects: Diagonal P-splines (penalties along age-year diagonals representing birth cohorts) capture cohort structure — e.g., the UK "golden cohort" born ca. 1925–1935 — invisible to age-period models.
• Uncertainty: Approximate posterior credible intervals for forecasts derive from the GLM Hessian at the penalized likelihood solution (the IRLS fixed point).

How It Works

Given a mortality array with deaths $d_{xt}$ and exposures $e_{xt}$:

1. Specify B-spline bases $\mathbf{B}_x$ (age) and $\mathbf{B}_t$ (year) with enough knots for flexibility; typically $c_x \times c_t$ coefficients in $\boldsymbol{\Theta}$.
2. Construct difference-penalty matrices $P_x = D_x^\top D_x$ and $P_t = D_t^\top D_t$ from second differences of adjacent row/column coefficients.
3. Estimate $\boldsymbol{\Theta}$ via penalized IRLS using Kronecker product operations (GLAM structure) on the coefficient array.
4. Forecast by extending $\mathbf{B}_t$ to future years $t > t_n$; the year penalty ensures smooth continuation.
5. Obtain prediction intervals from the posterior variance of $\boldsymbol{\Theta}$ propagated through the B-spline basis evaluation.

Why It Matters

P-spline mortality smoothing replaces the two-stage Lee-Carter pipeline (singular value decomposition (SVD) + autoregressive integrated moving average (ARIMA) on $k_t$) with a single unified model, handles irregular data (missing years, partial cohorts), and naturally incorporates cohort effects through diagonal smoothing. It is now standard in the StMoMo R package used by UK and European actuaries for Solvency II longevity reserve calculations. The method also serves as the foundation for the Renshaw-Haberman (2006) age-period-cohort extension and the Plat (2009) cohort model.

Open Questions

• Extrapolation has no structural interpretation — the implicit future mortality path is "approximately polynomial," lacking the Brownian motion interpretation of Lee-Carter's random walk with drift; it is unclear which assumption is more appropriate over long horizons.
• Smoothing parameters optimized by Akaike information criterion (AIC) or cross-validation for in-sample fit do not necessarily minimize forecast error.
• Diagonal cohort penalties can overfit short cohort diagonals observable only near the data boundary.

Related

• Lee-Carter Model — main comparison; rank-1 bilinear surface; P-splines nest it as a special case
• Iain D. Currie
• Paul H.C. Eilers
• María Durbán

Sources

• Currie Durban and Eilers 2004 — Smoothing and Forecasting Mortality Rates