Zellner's g-Prior

Definition

Zellner's $g$-prior is a conjugate prior for the coefficients of a normal linear regression that borrows the data's own design geometry: $\beta\mid\sigma^2 \sim N\!\big(0,\; g\,\sigma^2 (X'X)^{-1}\big)$, with a single scalar $g$ controlling how informative the prior is. Its analytical tractability — closed-form marginal likelihoods and Bayes factors that depend on the data only through $R^2$ and dimensions — made it the workhorse prior for Bayesian variable selection and model comparison. Maruyama-George (2011) generalize it to allow $p>n$ and make it fully Bayes by putting a prior on $g$ (Maruyama-George 2011).

Key Ideas

• Design-aligned covariance. Taking the prior covariance proportional to $(X'X)^{-1}$ means the prior "matches" the likelihood's information geometry, so the posterior mean is a simple shrinkage of the least-squares estimate: $\hat\beta_{\text{post}}=\tfrac{g}{1+g}\hat\beta_{\text{OLS}}$ — a ridge-type shrinkage toward zero governed by $g/(1+g)$.
• Closed-form marginal likelihood. Under the $g$-prior the marginal likelihood of a submodel depends on the data only through its $R^2$ and its dimensions, giving Bayes factors in closed form — the property behind fast Bayesian variable selection and model selection.
• The role of $g$. Large $g$ ≈ a diffuse prior (weak shrinkage); small $g$ ≈ strong shrinkage toward the null. Because $g$ scales the whole prior, the Bayes factor is highly sensitive to it.
• Fixed-$g$ paradoxes. A single fixed $g$ (e.g. $g=n$, "unit information") suffers pathologies: the information (Bartlett) paradox — as the evidence for a model grows without bound (data $R^2\to1$), the Bayes factor stays bounded — and inconsistency in some limits, echoing the prior-sensitivity concerns of Bayes factors generally.
• Fully Bayes / hyper-$g$. Placing a prior on $g$ and integrating it out (the hyper-$g$ priors of Liang et al. 2008; the generalized $g$-prior of Maruyama-George 2011) resolves the information paradox and restores model-selection consistency, while — in the Maruyama-George formulation — extending the prior to the $p>n$ regime via a singular-value-decomposition/ridge construction and retaining closed-form Bayes factors.

Why It Matters

• The default prior for Bayesian regression model choice. Its closed forms make exhaustive or MCMC model search feasible, underpinning variable selection, spike-and-slab search, and Bayesian model averaging.
• Turns prior choice into one interpretable knob. Reducing prior elicitation to a single $g$ (or a prior on $g$) is what made objective Bayesian model comparison practical.
• A bridge to ridge regression. The $g/(1+g)$ shrinkage exposes the $g$-prior posterior as Bayesian ridge, connecting model selection to regularized estimation.

Open Questions

• Choice of the hyper-prior on $g$ and its influence on selection in finite samples.
• Behaviour and calibration in high dimensions ($p\gg n$) beyond the Maruyama-George construction.
• Robustness of consistency results under model misspecification and correlated designs.

Related

• Bayes Factor
• Bayesian Model Selection
• Variable Selection
• Spike-and-Slab Prior
• Bayesian Linear Regression
• Maruyama-George (2011) Fully Bayes Factors with a Generalized g-Prior
• Liang et al. (2008) Mixtures of g-Priors for Bayesian Variable Selection
• Yuzo Maruyama
• Edward I. George
• Feng Liang
• James O. Berger
• Merlise Clyde