Bayesian Semiparametric Regression

Definition

Bayesian semiparametric regression estimates a model that combines a parametric component with a nonparametric (flexible-function) component — canonically the partial linear model $y_i = z_i\gamma + f(x_i) + \varepsilon_i$ , where $z_i\gamma$ is linear and $f(\cdot)$ is an unknown smooth function — by placing a prior directly on the function (or its values) and computing a posterior. The prior over the function's smoothness plays the role that a bandwidth or basis choice plays in classical nonparametrics, and Bayes factors provide a unified route to comparing parametric against semiparametric specifications.

Key Ideas

Function values as parameters. Treat $\delta = (f(x_1),\dots,f(x_N))'$ as unknowns. Stacking with the parametric part, $y = W\theta + \varepsilon$ with $W=(Z,I_N)$ is over-parameterised ( $N+k>N$ ), so $\theta$ is not identified by the likelihood; the prior supplies identification (Koop-Poirier 2004).
Smoothness prior ≈ classical bandwidth. A prior that penalises $f(x_i)-f(x_{i-1})$ (data ordered in $x$ ) encodes the same assumption as Yatchew's differencing estimator or kernel smoothing; a single hyperparameter controls smoothness and is the Bayesian analogue of the bandwidth. This reframes the "priors sneak in information" objection — the classical methods make an equivalent smoothness assumption.
Natural-conjugate analytics. For the partial linear Normal model, a Normal-Gamma prior gives closed-form posteriors — no MCMC — in contrast to Dirichlet-process, wavelet, spline, or Gaussian-process Bayesian nonparametrics, which generally require posterior simulation.
Local averaging. Posterior means of the nonparametric effect exhibit a kernel-like local averaging property, making the "black box" interpretable.
Unified estimation and testing. Bayes factors compare parametric vs. semiparametric (or two semiparametric) models; constrained estimation is imposed by zero prior weight on constraint-violating regions (Geweke 1986).
Extensions. The natural-conjugate machinery carries to qualitative-choice (a semiparametric probit), censored/truncated models, non-Normal errors, and — via seemingly-unrelated / multiple-equation structure — to systems of equations.

How It Works

Order observations in the nonparametric covariate, express the model as an over-parameterised Normal regression, and impose a partially-informative smoothness prior whose single hyperparameter is selected by data-based / optimal-bandwidth-style criteria. For the Normal partial linear model the posterior is available analytically; for non-Normal or limited-dependent extensions a Gibbs/data-augmentation step is added. Model comparison proceeds by marginal likelihoods / Bayes factors rather than asymptotic tests, an advantage where classical asymptotics for nonparametric estimators are complicated.

Multiple-equation systems (Koop-Poirier-Tobias 2003)

The single-equation construction generalizes to systems: a semiparametric SUR model $y_{ij}=z_{ij}'\beta_j+f_j(x_{ij})+\varepsilon_{ij}$ ( $j=1,\dots,m$ equations) becomes, after stacking each equation's function values as parameters, a standard SUR — and, as a (restricted) reduced form, covers semiparametric simultaneous-equations models, unlocking nonparametric instrumental variables and endogeneity correction. One wrinkle: in the system the natural-conjugate prior is too restrictive (it constrains cross-equation structure), so an independent Normal-Wishart prior is used instead, which reintroduces Gibbs sampling — but each equation's analytic single-equation block keeps the empirical-Bayes search over the smoothing hyperparameters $\eta_j$ cheap. Illustrated on a two-equation returns-to-schooling model (wage nonparametric in tenure, schooling nonparametric in ability), which recovers a strongly nonlinear ability→schooling relation a quadratic would miss.

Why It Matters

Simplicity and transparency. Delivers nonparametric flexibility with textbook Normal-regression tools, lowering the barrier that (per Yatchew 1998) keeps applied economists from using nonparametric methods.
Finite-sample, conditional inference. Being fully Bayesian, it conditions on the observed data and avoids reliance on nonparametric asymptotics.
Bridges paradigms. Makes explicit the correspondence between a smoothness prior and a bandwidth, and between constrained Bayesian estimation and classical restricted nonparametrics.

Open Questions

Sensitivity of posterior inference to the smoothness hyperparameter and to the ordering/spacing of the nonparametric covariate.
Scaling to multivariate $x$ (curse of dimensionality) beyond additive structures.
Computational cost once the Normal-conjugate structure is broken (limited-dependent, heavy-tailed, or systems extensions).