Chen-Ibrahim-Yiannoutsos (1999) Prior Elicitation, Variable Selection and Bayesian Computation for Logistic Regression Models

Summary

The paper develops a fully Bayesian approach to variable selection in logistic regression, tackling the three obstacles that make Bayesian model selection hard: (1) specifying priors for the regression coefficients in every candidate model, (2) specifying a prior over the model space, and (3) computing the analytically intractable prior and posterior model probabilities. For (1) it proposes a class of informative priors elicited semi-automatically from a prior prediction $y_0$ for the response and a scalar precision $a_0$ quantifying confidence in that prediction (a historical-data / power-prior-style construction). For (2) it gives a matching informative prior on the model space. For (3) it develops novel algorithms that compute all prior and posterior model probabilities from Gibbs samples of a single model, with theoretical properties derived. The methods are illustrated on cancer and AIDS clinical-trial data.

Key Claims

• Informative prior for the coefficients via prediction elicitation. Rather than eliciting a prior directly on the high-dimensional $\beta$, the analyst specifies a prior prediction $y_0$ of the response vector (e.g. from historical data or expert judgment) and a scalar $a_0 \ge 0$ measuring its precision; these induce a proper, conjugate-like prior on $\beta$ for each model semi-automatically. Setting $a_0 = 0$ recovers a noninformative limit; larger $a_0$ weights the historical prediction more (paralleling the power-prior discounting).
• Informative prior on the model space. A default uniform prior over models is often unsatisfactory; the paper constructs a prior on the model space that is coherent with the coefficient prior and can encode a preference toward models compatible with the prior prediction $y_0$.
• Model probabilities from a single model's Gibbs output. The central computational contribution: prior and posterior model probabilities (ratios of normalizing constants) for all submodels are estimated from Monte Carlo draws under one full model, via importance-weighting-style identities (building on Chen 1994), avoiding a separate run per model. Theoretical properties of the estimators are established.
• Practical logistic-regression variable selection. The combined scheme gives interpretable posterior model probabilities and is demonstrated on real cancer and AIDS trial datasets, contrasting with criterion-based selection (AIC, BIC) and with reference-prior approaches that can yield ambiguous Bayes factors.

Concepts Introduced or Extended

• Variable Selection — fully Bayesian variable selection for logistic regression with informative priors and computed model probabilities
• Power Prior — the prediction-based ($y_0$, $a_0$) informative prior is a historical-data construction in the power-prior family
• Bayesian Model Selection — priors on the model space; posterior model probabilities vs. Bayes factors / AIC / BIC
• Bayesian Logistic Regression — the base model
• Gibbs Sampler — single-model Gibbs output reused to compute all model probabilities

Entities Mentioned

• Ming-Hui Chen
• Joseph G. Ibrahim
• Constantin Yiannoutsos

Quotes

"We examine the problem of eliciting informative prior distributions for the regression parameters as well as the elicitation of an informative prior distribution for the model space for Bayesian selection of variables in logistic regression."

"Another major contribution of this paper is to develop novel computational algorithms for computing analytically intractable prior and posterior model probabilities. These algorithms are efficient and only require Gibbs samples from a single model."

My Take

This is the logistic-regression companion to the Ibrahim-Chen programme on informative priors from historical data, and it pairs naturally with the Chen-Ibrahim-Sinha (1999) cure-rate paper. The clever moves are (a) shifting elicitation from the opaque coefficient space to an interpretable prediction $y_0$ with a single confidence knob $a_0$, and (b) computing every submodel's marginal probability from one model's MCMC output — a big practical saving when the model space is large. It sits outside the wiki's time-series core but connects to the existing variable-selection (SSVS / George-McCulloch) and power-prior material, and reuses the same Gibbs/normalizing-constant machinery (Chib 1995, bridge/importance sampling) that the econometrics side relies on.