Zellner-Min (1995) Gibbs Sampler Convergence Criteria

Summary

Zellner and Min present three operational Gibbs Sampler Convergence Criteria (GSC$^2$) for the Gibbs sampler (GS) that detect not merely convergence failure but convergence to an incorrect distribution — a gap left open by Gelman-Rubin and Raftery-Lewis diagnostics. The criteria exploit the fact that when the joint posterior $p(\alpha,\beta|D)$ is known up to a constant, the ratio $p(\alpha|D)p(\beta|\alpha,D) = p(\beta|D)p(\alpha|\beta,D)$ must hold everywhere, enabling a direct check against GS-estimated marginals. All three criteria are illustrated on a simple regression, a two-equation Seemingly Unrelated Regression (SUR), and a bimodal mixture that traps the GS near one mode.

Key Claims

• Existing convergence diagnostics (Gelman-Rubin 1992, Raftery-Lewis 1992) tell you whether the chain has mixed — not whether it has converged to the correct target. A GS trapped in one mode of a bimodal posterior can appear "converged" under these tests.
• ARC$^2$ (Anchored Ratio Criterion): $\theta_{12} = p(\alpha_1,\beta_1|D)/p(\alpha_2,\beta_2|D)$ is computable exactly from the known joint posterior form (constants cancel). Compare to the GS estimate $\hat\theta_{12}(N)$ built from estimated marginals; large discrepancy flags failure.
• DC$^2$ (Difference Criterion): From the identity $p(\alpha|D)p(\beta|\alpha,D) = p(\beta|D)p(\alpha|\beta,D)$, define $\hat\eta_i(N) = \hat{p}_N(\alpha_i|D)p(\beta_i|\alpha_i,D) - \hat{p}_N(\beta_i|D)p(\alpha_i|\beta_i,D)$. Should be near zero if converged correctly; evaluated at multiple parameter-space points.
• RC$^2$ (Ratio Criterion): $\hat\gamma_i(N) = \hat{p}_N(\alpha_i|D)p(\beta_i|\alpha_i,D) / \hat{p}_N(\beta_i|D)p(\alpha_i|\beta_i,D)$. Should be near 1.
• Bimodal failure case: For the 10/90 mixture $p = 0.1 p_A + 0.9 p_B$ (modes at $(0,5)$ and $(0,-5)$), GS correctly "converges" near $(0,5)$; DC$^2$ gives $\hat\eta = -0.509$ at $(0,-5)$, clearly flagging that the GS estimates are wrong — even though the chain appears stationary.
• Simple regression ($T=20$, $\sigma^2=4$): DC$^2$ mean $|\hat\eta| = 4.6\times10^{-5}$ at $N=1{,}000$; $5.6\times10^{-6}$ at $N=9{,}000$. RC$^2$ departs from 1 by up to $8$–$9\%$ at $N=1{,}000$ but within $0.5\%$ at $N=9{,}000$. GS marginals converge to exact posteriors (Table 1).
• SUR (two-equation, $T=20$): DC$^2$ $\hat\eta_1$ and $\hat\eta_2 \to 0$ as draws increase; posterior odds $K_{OA}$ for $H_0\colon \eta=0$ rise from $10.98$ at $N=100$ to $118.68$ at $N=9{,}000$ — strongly favoring the null and confirming correct convergence.
• Inference appendix: $\hat\eta_i(N) \approx \mathcal{N}(\eta_i, \hat\phi_i^2)$ asymptotically; variance formula (eq. 12) accounts for GS autocorrelation. Posterior odds: $K_{OA} = \sqrt{\pi N/2}(1+z^2/N)\exp\{-z^2/2\}$ (Jeffreys-based Cauchy prior). Vector extension: $m$ points simultaneously, $(\hat\eta-\eta)'\hat\Sigma^{-1}(\hat\eta-\eta) \approx \chi^2_{N-m}$.

Concepts Introduced or Extended

• Gibbs Sampler

Entities Mentioned

• Arnold Zellner
• Chung-Ki Min

Quotes

"It does not appear that operational convergence criteria that indicate not only convergence of the GS but also convergence to a correct result have appeared in the literature."

"The GS estimated marginals have converged to a correct values… the null hypothesis is favored" [SUR example, $K_{OA} = 118.68$ at $N=9{,}000$].

My Take

The paper fills a genuine gap: standard convergence diagnostics are blind to the failure mode where the GS gets stuck in the wrong part of the parameter space and produces a stationary but incorrect chain. The three GSC$^2$ criteria are cheap to compute whenever the joint posterior $p(\alpha,\beta|D)$ form is known analytically — which is exactly the setting where Gibbs sampling is used. The bimodal example is the clearest demonstration. One limitation: the criteria require explicit knowledge of $p(\alpha,\beta|D)$ up to a constant and the conditional densities in closed form, so they are not applicable to fully non-conjugate settings where the GS itself is being used precisely because no analytical form is available.