Probabilistic Population Forecasting

Definition

Probabilistic population forecasting generates probability distributions over future population trajectories by embedding stochastic vital rate models within the cohort-component demographic projection framework (Leslie matrix), replacing deterministic high/medium/low scenario projections with prediction intervals whose probability content can be stated and empirically evaluated. All demographic forecasting methods fall into one of three approaches (Booth 2006): extrapolation (project observed past trends forward with no causal assumptions), expectation (individual surveys of intended behavior or structured expert judgment), and structural (causal modeling of demographic drivers). Most operational practice is extrapolative; structural approaches face fundamental limitations at long horizons.

Key Ideas

Cohort-component method: The standard population projection engine. A Leslie matrix $L(t)$ advances a population vector $N(t)$ one period forward via $N(t+1) = L(t) \cdot N(t)$ , where $L(t)$ embeds age-specific survival probabilities and fertility rates. Stochastic population forecasting introduces random variation in $L(t)$ and propagates the induced uncertainty through repeated matrix multiplication. Two implementation strategies exist: analytic propagation of error (Alho and Spencer 1985 approximately linear model: $V(t+1) = R(t)V(t)$ ) and Monte Carlo simulation (Lee-Tuljapurkar 1994 random sample paths).
Factor hierarchy for mortality models: Zero-factor models extrapolate each age-specific mortality rate independently at its own historical trend. One-factor models (Lee-Carter) capture the dominant trend via a single latent index $k(t)$ ; the first singular value pair typically explains $>90\%$ of log-rate variance. Two-factor models allow young and old ages to diverge (as during the opioid crisis). Three-factor age-period-cohort (APC) models add cohort effects. Parsimony and explanatory power trade off across the hierarchy; the single-factor Lee-Carter (LC) structure is cross-nationally robust across all G7 countries (Tuljapurkar, Li, and Boe 2000).
Three uncertainty quantification approaches: Three complementary methods for bounding forecast uncertainty are now recognized as producing broadly consistent estimates (Booth 2006):
1. Ex ante model-based: stochastic time series (Lee-Carter, autoregressive moving average (ARMA)) generating prediction intervals analytically or by simulation. Intervals grow as $\sigma\sqrt{h}$ for random-walk processes.
2. Ex ante expert-based: structured elicitation of probability distributions for future vital rates from domain experts; implemented in International Institute for Applied Systems Analysis (IIASA) world population projections (Lutz et al.). Avoids reliance on historical data patterns but subject to assumption drag.
3. Ex post error-based: empirical distribution of historical forecast errors serves as an uncertainty benchmark. Keilman (1997) and Alho's naïve-error approach use this to evaluate whether model-based intervals achieve claimed coverage rates.
Assumption drag: Expert opinion on future mortality systematically underestimates the pace of improvement (Alho and Spencer 1990). Expert panels anchor on recent experience and lag behind realized trends; confirmed for fertility as well (Lee 1999). The pattern is structural rather than incidental: institutional incentives and cognitive anchoring both push experts toward conservative projections.
Ashley's impossibility theorem (1983): Including an exogenous variable in a structural demographic model improves forecast accuracy only if the model's forecast mean squared error (MSE) for that variable is itself less than the variable's unconditional variance — i.e., only if the model forecasts that driver better than a naïve "predict the mean" baseline. The probability that this condition holds decreases as the forecast horizon lengthens. This provides a principled theoretical explanation for why structural demographic models (fertility as a function of income, education, or policy) have not outperformed extrapolative models in out-of-sample evaluation.
Accuracy non-improvement: Despite twenty-five years of methodological development and proliferating approaches, "no evidence that overall accuracy in mortality and fertility forecasting has improved over time" (Keilman 1997). Realized forecast errors have not diminished systematically across the review period. Methods that are more principled — e.g., generating well-calibrated probability intervals — may be better even if their point-forecast root mean squared error (RMSE) is unchanged.
Probabilistic consistency: A critical advance of the period. Earlier practice generated separate stochastic models for mortality and fertility, then combined them in a cohort-component projection without ensuring that the joint distribution was coherent (e.g., correlated fertility and mortality shocks). Probabilistic consistency — using the same Monte Carlo sample paths for all vital rates in a single projection — avoids this incoherence and is now standard. See Lee-Carter Model and Long-Term Actuarial Balance for applications.
Cause-of-death disaggregation: Consistently found to reduce, not improve, forecast accuracy (Booth 2006 synthesis). Disaggregated projections are always higher than aggregate projections — i.e., they overestimate all-cause mortality and underestimate life expectancy — because separate forecasts for each cause lose the negative cross-cause correlations (Wilmoth 1995). This is a mathematical result, not an empirical observation.

How It Works

Leslie matrix propagation: Partition the population by single-year age groups. The age-1 entry of $N(t+1)$ is the sum of births from each fertile age group (product of age-specific fertility rates $\times$ female population $\times$ sex ratio at birth). The age- $(x+1)$ entry is the age- $x$ entry of $N(t)$ multiplied by the age- $x$ survival probability. Repeat for 100+ age groups.

Stochastic vital rates: In the Lee-Tuljapurkar (1994) Monte Carlo framework, each projection year draws a random innovation to $k(t)$ from $N(0, \sigma^2)$ . All age-specific death rates in that year are updated via their $b(x)$ loadings. A separate ARMA model drives total fertility rate (TFR). $750$ sample paths were used in Lee and Tuljapurkar (1998) to characterize the distribution of Old-Age, Survivors, and Disability Insurance (OASDI) 75-year solvency outcomes.

Analytic propagation: Alho and Spencer's (1985) approximately linear model propagates a covariance matrix through the projection rather than simulating sample paths. Faster but requires linearity assumptions that may fail when uncertainty is large.

Why It Matters

Scenario approach is insufficient: Scenarios specify deterministic trajectories for vital rates and produce no probability statements. They cannot distinguish whether the "high" scenario is a 95th or 75th percentile outcome, making it impossible to assess policy robustness or compute actuarial solvency probabilities.
Social Security solvency: The first fully probabilistic OASDI solvency analysis (Lee and Tuljapurkar 1998) found a mean 75-year deficit of $-3.3$ percentage points (pp) vs. Social Security Administration (SSA)'s $-2.2$ pp using deterministic methods. Fertility structural uncertainty dominates 75-year uncertainty; mortality uncertainty is secondary. See Long-Term Actuarial Balance.
Expert opinion benchmarking: Once model-based intervals are computed, ex post evaluation can determine whether official high-low ranges achieve their implicit coverage claims. Studies consistently find that official ranges are too narrow for a $67\%$ or even $50\%$ interpretation (Alho and Spencer 1985; Keilman 1997).

Open Questions

Why has forecast accuracy not improved despite the proliferation of methods? Is accuracy bounded by fundamental unpredictability of demographic change, or are model-building practices failing to incorporate available information?
Can the three uncertainty approaches (model-based, expert-based, ex post error) be formally combined into a single posterior? What prior over model space is implied by equal weighting?
Does assumption drag in expert judgment partially compensate for model-based underestimation of structural breaks? Or do both errors compound?
How do probabilistic population forecasts perform in the context of disruptive demographic events (COVID-19, opioid crisis, rapid fertility change) that lie outside the historical sample?