Koop-Poirier (2004) Bayesian Variants of Some Classical Semiparametric Regression Techniques

Summary

Koop and Poirier develop a Bayesian treatment of the partial linear model $y_i = z_i\gamma + f(x_i) + \varepsilon_i$ that needs nothing beyond the standard Normal linear regression model with a natural-conjugate prior — no MCMC. The trick is to treat the unknown-function values $\delta=(f(x_1),\dots,f(x_N))'$ as parameters and impose a smoothness prior that mirrors the classical differencing estimator (Yatchew 1998), which delivers closed-form posteriors, kernel-like local averaging, and Bayes factors for testing parametric against semiparametric specifications. Extensions cover semiparametric probit, non-Normal errors, and additive models.

Key Claims

• Reframe the partial linear model as an over-parameterised Normal regression. With $W=(Z,I_N)$ and $\theta=(\gamma',\delta')'$, $y=W\theta+\varepsilon$ has more unknowns ($N+k$) than observations, so $\theta$ is unidentified and $W'W$ singular; identification comes entirely from the prior, and $\delta\perp y\mid W\theta$.
• Smoothness prior = Bayesian differencing. The prior embeds that $f(x_i)-f(x_{i-1})$ (with data ordered $x_1\le\cdots\le x_N$) is small — the exact assumption behind Yatchew's differencing estimator (bounded first derivative + finite support ⇒ consistency of OLS on differenced data). One variant makes the posterior mean of $\gamma$ equal the OLS estimate on differenced data; the idea extends to any order of differencing.
• Natural-conjugate analytics, no MCMC. A Normal-Gamma prior on $(\gamma,\delta,\sigma^{-2})$ yields fully analytical posterior results, sidestepping the "computationally intensive" objection to nonparametrics; unlike Dirichlet-process / wavelet / spline / Gaussian-process Bayesian nonparametrics, no difficult posterior simulation is required for the partial linear Normal case.
• Single interpretable hyperparameter. Only one smoothness hyperparameter $\eta$ is elicited; it plays the role of the classical bandwidth, and the paper discusses optimal/data-based choices. The posterior means exhibit a local averaging property analogous to kernel smoothing — "looking inside the Bayesian black box."
• Unified estimation + testing. Bayes factors compare a parametric model to a semiparametric alternative (or two semiparametric models) straightforwardly; constrained estimation is handled by zero prior weight on constraint-violating regions (Geweke 1986). Connections drawn to extreme-bounds analysis.
• Extensions. The framework carries over — at some computational cost — to qualitative-choice, censored/truncated models (a fully worked semiparametric probit) and relaxes the Normal-error assumption.

Concepts Introduced or Extended

• Bayesian Semiparametric Regression

Entities Mentioned

• Gary Koop
• Dale J. Poirier

Quotes

"The simplicity of our theoretical results allows us to look inside the Bayesian black box and understand what precisely our methods are doing (e.g. we show that the Bayesian posterior means of nonparametric effects exhibit a local averaging property analogous to kernel smoothing)."

"The only type of prior input required is one prior hyperparameter, η, which controls the degree of smoothness of f(·)."

My Take

The appeal here is deflationary in the best sense: instead of importing Dirichlet processes or Gaussian-process priors, Koop and Poirier show that the classical partial linear model is just a natural-conjugate Normal regression whose non-identification is resolved by a smoothness prior that literally encodes Yatchew's differencing assumption — so you inherit textbook analytics, Bayes-factor model comparison, and a single bandwidth-like hyperparameter, all without MCMC. The honest caveat they flag is a terminological one worth keeping: their finite-sample models have a finite-dimensional parameter space, so they are not "semiparametric" in the strict sense — they adopt the classical label to highlight the mapping. The deeper contribution to the wiki is conceptual: it makes explicit that a prior over smoothness is the Bayesian counterpart of a bandwidth choice, dissolving the usual "Bayesian methods sneak in prior information" objection since the classical kernel/differencing methods make an equivalent assumption. It pairs directly with Koop, Poirier and Tobias (2003), which pushes the same smoothness-prior machinery into multiple-equation (SUR / simultaneous-equations) systems.