Maruyama-George (2011) Fully Bayes Factors with a Generalized g-Prior

Summary

For the normal linear model variable-selection problem, Maruyama and George propose model-selection criteria based on a fully Bayes formulation using a generalization of Zellner's $g$-prior that allows for $p>n$. A special case of the prior yields tractable closed forms for marginal densities and Bayes factors, revealing new model-evaluation characteristics of potential interest. The construction leans on ridge regression and the singular value decomposition, and delivers model-selection consistency.

Key Claims

• Fully Bayes. Rather than fix the $g$-prior scale, they place a prior on $g$ and integrate it out, avoiding the pathologies of fixed-$g$ Bayes factors (the information/Bartlett paradox) and restoring desirable behaviour.
• Generalized $g$-prior for $p>n$. They generalize Zellner's $g$-prior — whose usual form requires $X'X$ invertible — to the high-dimensional $p>n$ regime via a singular-value-decomposition / ridge construction.
• Closed forms. A special case of the prior gives tractable closed-form marginal densities and Bayes factors for submodels, keeping model search computationally feasible.
• Model-selection consistency. The resulting criteria are consistent for selecting the true submodel under appropriate conditions.
• New evaluation characteristics. The closed forms expose model-evaluation features (through $R^2$, dimensions, and the SVD/ridge structure) of interest for comparing models.

Concepts Introduced or Extended

• Zellner's g-Prior
• Bayes Factor

Entities Mentioned

• Yuzo Maruyama
• Edward I. George

Quotes

"We propose selection criteria based on a fully Bayes formulation with a generalization of Zellner's g-prior which allows for p > n. A special case of the prior formulation is seen to yield tractable closed forms for marginal densities and Bayes factors which reveal new model evaluation characteristics of potential interest."

My Take

A clean fix to the two things that limited the workhorse $g$-prior: its sensitivity to a fixed $g$ (which breeds the information paradox) and its reliance on $X'X$ being invertible (which rules out $p>n$). Making the prior fully Bayes — a prior on $g$, integrated out — is the same "don't commit to a scale" move that the hyper-$g$ literature (Liang et al. 2008) uses, and it is the constructive antidote to the Bayes-factor prior-sensitivity worries the wiki catalogs elsewhere (up to and including Lovric's reversal paradox). The SVD/ridge route to $p>n$ is what makes it genuinely modern, and the payoff — retaining closed-form Bayes factors while gaining consistency — is why it belongs alongside Kass-Raftery and Raftery (1995) as machinery for practical Bayesian model selection.