Albert (1996) Bayesian Selection of Log-Linear Models

Summary

Develops a Bayesian framework for selecting among log-linear models for contingency tables using a Poisson likelihood. The key prior innovation is a block-diagonal precision matrix $\Sigma^{-1} = \text{diag}(0I_{p_0}, P_1I_{p_1},\ldots,P_sI_{p_s})$ with Gamma hyperpriors on the group precisions $P_j$ ; choosing shape parameter $\nu_j=1$ gives Cauchy marginals for the variable-specific coefficients, producing more robust model selection than the Normal alternative. Posterior model probabilities are computed via a Laplace approximation with an iteratively weighted least squares (IWLS) mode-finder, and Bayesian model averaging (BMA) is implemented via a Gibbs sampler over $(\{\beta_j\}, Z, P^*)$ .

Key Claims

The block-diagonal precision prior $\Sigma^{-1} = \text{diag}(0I_{p_0}, P_1I_{p_1},\ldots,P_sI_{p_s})$ places a flat (improper) prior on the always-included intercept/structural parameters $\beta_0$ and a Normal prior with precision $P_j$ on each group of testable parameters $\beta_j$ .
Integrating out $\beta_j$ over the Gamma( $\nu_j/2$ , $b_j^2\nu_j/2$ ) hyperprior on $P_j$ yields a $t_{\nu_j}$ marginal; setting $\nu_j=1$ gives Cauchy marginals, which have heavier tails and are more robust to large coefficient values than the Normal ( $\nu_j \to \infty$ ).
Posterior model probabilities are approximated via the Laplace method (eq. 3): $\text{Pr}(M_k|y,P) \approx C_k\,\text{Pr}(M_k)\,f(y|\bar\lambda_k)\,|\Sigma_0|^{-1/2}\exp\!\left(-\tfrac{1}{2}\bar\lambda_k'\Sigma_0^{-1}\bar\lambda_k\right)|\bar\Sigma_0|^{1/2}$ where $\bar\lambda_k$ is the posterior mode found by IWLS: $(X_0'WX_0+\Sigma_0^{-1})\lambda_k = X_0'WZ$ with $W=\text{diag}(\bar{m}_0)$ and $Z=\bar\eta_0+(y-\bar{m}_0)/\bar{m}_0$ .
Gibbs sampler cycles over three blocks: (1) $\beta_j|\text{rest}$ — conjugate Normal given $P_j$ ; (2) $Z|\text{rest}$ — auxiliary working response; (3) $P_j^*|\text{rest}$ — Gamma( $\nu_j/2+p_j/2$ , $b_j^2\nu_j/2+\sum_t\beta_{jt}^2/2$ ).
Bayesian model averaging inflates posterior standard errors by nearly 2× relative to the maximum-probability single model, reflecting genuine model uncertainty in the two applications.
Cauchy prior is more robust than Normal when a few cells are outlying (extreme counts); Normal prior collapses to a point-mass model when precision $P_j$ is driven very large.
Connection to George-McCulloch (1993): both use hierarchical Normal/Gamma priors over coefficient groups to search model space; Albert (1996) extends the idea to Poisson log-linear models and replaces the stochastic search variable selection (SSVS) indicator with a continuous Laplace approximation to marginal likelihoods.

Concepts Introduced or Extended

Entities Mentioned

Quotes

"The Cauchy prior is more robust than the normal prior in the sense that the Cauchy prior assigns relatively more probability to large values of the regression coefficients."

"In the Bayesian model averaging approach, the posterior standard deviations of fitted values are on average almost twice as large as the standard deviations from the single best model."

My Take

This is a natural and technically clean extension of the George-McCulloch SSVS framework to discrete (Poisson) responses. The Gamma hyperprior trick for inducing Cauchy marginals is elegant and generalizes to any exponential-family generalized linear model (GLM). The Laplace approximation works well for the log-linear setting because the Poisson log-likelihood is strictly concave, making the IWLS mode-finder reliable. The BMA-vs-single-model comparison is the most practically useful result: the near-doubling of standard errors is a compelling quantitative argument against reporting only the maximum a posteriori (MAP) model. The paper predates the modern Markov chain Monte Carlo (MCMC)-based BMA literature (e.g. Madigan-York 1995) but already delivers the key intuition about model uncertainty inflating inferential uncertainty.