Wahba (1978) Improper Priors, Spline Smoothing and Guarding Against Model Errors

Summary

Wahba shows that spline (and generalized spline) smoothing is equivalent to Bayesian estimation under a partially improper prior. This equivalence supports spline smoothing as a natural solution to regression when one is handed a set of regression functions but wants to hedge against the possibility that the true model does not lie exactly in their span. The spline theory yields a natural measure of the true model's deviation from that span, which can be estimated from the data and used to constrain the fit; convergence results and computational tricks are also given.

Key Claims

• Spline smoothing = Bayes with a partially improper prior. The smoothing-spline estimate coincides with the posterior mean under a flat (improper) prior on the parametric part — the span of the given regression functions (e.g. low-order polynomials) — and a proper Gaussian prior, induced by the roughness penalty, on the deviation from that span.
• Guarding against model error. Fitting only the specified regression functions is vulnerable if the truth is outside their span; adding the penalized nonparametric deviation hedges against this misspecification, making spline smoothing a robust default.
• A natural deviation measure. The spline theory supplies a natural measure of how far the true model departs from the span; an appropriate value can be estimated from the data and used to constrain the estimated model.
• Smoothing parameter as a variance ratio. The smoothing parameter corresponds to the ratio of the error variance to the prior (deviation) variance — the seed of later variance-component/REML approaches to choosing it.
• Convergence results and computational tricks accompany the theory.

Concepts Introduced or Extended

• Generalized Additive Model
• P-splines

Entities Mentioned

• Grace Wahba

Quotes

"Spline and generalized spline smoothing is shown to be equivalent to Bayesian estimation with a partially improper prior … a natural solution to the regression problem when one is given a set of regression functions but one also wants to hedge against the possibility that the true model is not exactly in the span of the given regression functions."

My Take

A foundational bridge: it makes precise the intuition that a roughness penalty is a prior, and specifically that the improper part corresponds to "let the parametric trend be whatever the data say" while the proper Gaussian part shrinks the wiggly deviation. That partial-improperness is exactly why later mixed-model/REML treatments of smoothing (Wood 2011) put diffuse priors on the polynomial null space and variance-component priors on the penalized range space. The framing as guarding against model error is the enduring lesson — smoothing is not just curve-drawing but a hedge against trusting a finite regression basis. It anchors the Bayesian side of the P-splines/GAM cluster.