Brown-Vannucci-Fearn (1998) Multivariate Bayesian Variable Selection and Prediction

Summary

Brown, Vannucci and Fearn extend Bayesian spike-and-slab variable selection from univariate to the multivariate regression model — selecting one subset of $p$ regressors that is good for predicting all $q$ responses jointly. A latent binary vector $\gamma\in\{0,1\}^p$ flags which coefficients are near zero versus not; under natural-conjugate (matrix-normal / inverse-Wishart) priors the coefficients and error covariance integrate out, giving a closed-form marginal posterior of $\gamma$. Fast add/delete updates make direct evaluation feasible in modest dimension, and MCMC over model space handles the high-dimensional case; prediction is by posterior model averaging over likely subsets. The motivating application is chemometric: predict the proportions of $q=3$ sugars from $p=160$ near-infrared absorbances.

Key Claims

• Multivariate generalization of George-McCulloch SSVS. The paper explicitly generalizes the univariate Gaussian SSVS of George-McCulloch (1993, 1997): the same latent-indicator machinery, but with $q$ correlated responses so a single $\gamma$ chooses regressors serving the whole response vector.
• Latent binary selection vector. $\gamma_j=1$ if regressor $j$'s (row of) coefficients is "not close to 0", $0$ otherwise; a prior on $\gamma$ (e.g. independent Bernoulli, expected model size fixing the hyperparameter) induces the model prior.
• Closed-form marginal posterior of $\gamma$ under conjugacy. With matrix-normal coefficients and inverse-Wishart error covariance (natural conjugate), integrate out the coefficients and $\Sigma$ analytically, leaving $p(\gamma\mid Y)$ up to a normalizing constant — the object to search or sample.
• Fast computation + MCMC. Rank-one add/delete updates make sequential evaluation cheap; for $p$ in the hundreds a Metropolis sampler over the $2^p$ model space approximates $p(\gamma\mid Y)$, visiting high-probability subsets.
• Prediction by model averaging. The posterior predictive distribution for new responses is obtained by averaging the model-specific predictives over the sampled subsets (Bayesian model averaging), rather than conditioning on a single selected model.
• Chemometric application. On near-infrared spectroscopy data the method picks informative wavelengths and predicts three sugar concentrations, a $p\gg$ interpretable-signal setting where selecting and averaging both matter.

Concepts Introduced or Extended

• Variable Selection
• Spike-and-Slab Prior

Entities Mentioned

• Philip J. Brown
• Marina Vannucci
• Tom Fearn

Quotes

"A latent vector with $p$ binary entries serves to identify one of two types of regression coefficients: those close to 0 and those not."

"Specializing our general distributional setting to the linear model with Gaussian errors and using natural conjugate prior distributions, we derive the marginal posterior distribution of the binary latent vector."

My Take

This is the paper that carries spike-and-slab selection into the multivariate, high-dimensional world, and its two moves are the ones that made it a chemometrics/genomics staple. First, using a single selection vector $\gamma$ for all $q$ responses is the right modeling choice when the responses share a common set of relevant predictors (sugars sharing informative NIR bands, genes sharing regulators) — it borrows strength across responses instead of running $q$ separate selections. Second, exploiting conjugacy to marginalize the coefficients and covariance turns selection into a search over the binary $\gamma$ alone, which is what makes $p=160$ (and later $p$ in the thousands) tractable by Metropolis-over-models plus fast rank-one updates. It sits directly on top of the wiki's SSVS and spike-and-slab pages as their multivariate case, and its predict-by-averaging stance — never commit to one subset — is the same lesson the BMA thread teaches. The companion Brown-Fearn-Vannucci (1999) paper refines the choice of variables (decision-theoretic subset selection) on top of this machinery.