O'Hagan (1995) Fractional Bayes Factors for Model Comparison

Summary

Introduces the fractional Bayes factor (FBF) as a principled solution to the problem of Bayesian model comparison under improper (vague) priors. When priors are improper, standard Bayes factors are indeterminate: unspecified normalizing constants $c_i$ do not cancel. The FBF avoids this by expressing the model-comparison update in terms of fractional likelihoods $f_i(x|\theta_i)^b$ and $f_i(x|\theta_i)$, where $b \in (0,1)$ is a "training fraction." The constants cancel within each model's ratio. The result is consistent, analytically tractable for linear models, and asymptotically equivalent to the Bayesian information criterion (BIC) when $b = n^{-1}$. Published with extended discussion in the Journal of the Royal Statistical Society, Series B (JRSS-B).

Key Claims

• The improper prior problem. Under model $M_i$ with improper prior $\pi_i(\theta_i) = c_i h_i(\theta_i)$, the marginal likelihood $q_i(x) = c_i \int h_i(\theta_i) f_i(x|\theta_i)\, d\theta_i$. The Bayes factor (BF) $B_{12}(x) = q_1(x)/q_2(x) \propto c_1/c_2$ — indeterminate without fixing the ratio of normalizing constants.
• Partial Bayes factors (Section 1.4). Split data $x$ into training sample $y$ (size $m$) and remainder $z$. Use $y$ to form proper posterior $\pi_i(\theta_i|y)$; the partial BF $B(z|y) = q_1(z|y)/q_2(z|y)$ is then well-defined since $c_i$ cancels. Identity: $B(x) = B(y)\cdot B(z|y)$. Problem: $\binom{n}{m}$ choices of training sample give different answers.
• Fractional Bayes factor (FBF, Section 2.3). Set $b = m/n$; a fraction-$b$ subsample has likelihood $\approx f_i(x|\theta_i)^b$. Define $$q_i(b,x) = \frac{\int h_i(\theta_i)\, f_i(x|\theta_i)\, d\theta_i}{\int h_i(\theta_i)\, f_i(x|\theta_i)^b\, d\theta_i}, \qquad B_b(x) = \frac{q_1(b,x)}{q_2(b,x)}.$$ Constants $c_i$ cancel in each $q_i(b,x)$. No training-sample enumeration.
• Asymptotics. $-2\log B_b(x) \approx (1-b)\bigl[-2\log(L_1/L_2) + (p_1-p_2)\,c(b)\bigr]$, where $c(b) = -\log b/(1-b)$. Consistency requires $b \to 0$ as $n \to \infty$ (so $c(b) \to \infty$); BF $\to \infty$ if model 1 true, $\to 0$ if model 2 true.
• Choice of $b$. Recommended: $b = m/n$ where $m$ is the minimal training sample size — the smallest $m$ making all model posteriors proper. For a model with $p$ parameters, $m = p$. Often taken as $b = n^{-1}$ in practice.
• Linear models (Section 4). For two normal linear models with improper prior $h_i(\theta_i) \propto \sigma_i^{-2}$, the FBF has closed form: $$B_b(x) = \frac{\Gamma(n/2)\,\Gamma\{(nb-r_2)/2\}}{\Gamma(nb/2)\,\Gamma\{(n-r_1)/2\}} \cdot \left(\frac{S_2^2}{S_1^2}\right)^{(n-r_1)/2}$$ where $r_i = p_i - 2t_i + 1$ and $S_i^2$ is the residual sum of squares. No numerical integration needed; works for non-nested as well as nested models.
• BIC connection. With $b = n^{-1}$, the FBF asymptotically equals the Schwarz BIC criterion. Numerically confirmed on the Darwin data: FBF = 3.814, BIC approximation = 3.810.
• Prior sensitivity (Section 5). Sensitivity of $B_b$ to prior misspecification is $O(b^{-1})$ — decreasing as $b$ increases. Larger $b$ gives less sensitivity but also costs consistency; calibrated trade-off.
• vs. Intrinsic Bayes factor (Berger-Pericchi 1993). IBF averages partial BFs over all minimal training samples (expensive; also not obviously coherent). The FBF achieves the same goal analytically and avoids the averaging problem. O'Hagan argues FBF is more coherent.
• vs. Posterior Bayes factor (Aitkin 1991). Uses $a=1, b$ in the generalized formulation $B_{a,b}(x)$. Not consistent: $c(a,b) = \log 2 = O(1)$, so posteriors do not concentrate on the true model. Specific critique documented in Section 2.3.

Concepts Introduced or Extended

• Bayes Factor

Entities Mentioned

• (none with existing wiki pages)

Quotes

"A new variant of the partial Bayes factor, the fractional Bayes factor, is advocated on grounds of consistency, simplicity, robustness and coherence."

My Take

A landmark paper for Bayesian model comparison. The core insight is clean: use fractional likelihoods to cancel improper-prior constants without picking a specific training sample. The linear-model formula is practically useful. The BIC connection (with $b = n^{-1}$) is an elegant justification for a familiar criterion from a fully Bayesian standpoint. Main weaknesses: (1) choice of $b$ still requires subjective input (what is "minimal"?); (2) the asymptotic motivation breaks down for small $n$; (3) for highly non-linear models the closed-form $q_i(b,x)$ may not be available. The published version includes a rich discussion by Berger, Rubin, Lindley, Aitkin, and others — essential reading for the debate between FBF, IBF, and posterior BF approaches.