Albert-Chib (1997) Bayesian Tests and Model Diagnostics in Conditionally Independent Hierarchical Models

Summary

Develops a unified Markov chain Monte Carlo (MCMC) framework for model criticism and testing within the conditionally independent hierarchical model (CIHM): $y_i \sim f(y_i|\theta_i)$ independently, $\theta_i \sim g(\theta_i|\lambda)$ independently, $\lambda \sim h(\lambda)$ . Three diagnostic perturbations of the base exchangeable model — scale-inflated outlier mixtures, partial exchangeability, and mixture link functions — are each implemented by adding a discrete latent indicator block to the standard MCMC cycle. A separate algorithm computes Bayes factors for the fixed-effects vs. random-effects hypothesis by simulating from a mixed prior on $\tau^2$ and reading off posterior probabilities from MCMC frequencies. Applied to cancer mortality rates for 84 Missouri cities and a 32-item mathematics placement test.

Key Claims

The base CIHM MCMC (§2.1) has three blocks: (1) $n$ independent Metropolis steps for $\theta_i$ with random-walk $N(0,c)$ increment ( $c \approx 2\sigma_{\text{target}}$ ); (2) conjugate Normal for $\beta|\{\theta_i\},\tau^2$ ; (3) conjugate inverse-gamma (IG) for $\tau^2|\{\theta_i\},\beta$ . For the basic exchangeable case ( $\theta_i \sim N(\mu,\tau^2)$ ), both $\mu$ and $\tau^2$ have closed-form conjugate conditionals.
Outlier model (§2.2): Replace $N(\mu,\tau^2)$ prior with scale-inflated mixture $(1-p_i)\phi(\theta_i;\mu,\tau^2) + p_i\phi(\theta_i;\mu,K\tau^2)$ , $K>1$ . Latent $A_i \in \{1,K\}$ is drawn Bernoulli at each iteration; $P(A_i=K|\text{rest}) \propto p_i\phi(\theta_i;\mu,K\tau^2)$ . Posterior probability $P(A_i=K|y)$ identifies outlying units. Cancer mortality ( $K=3$ , $p_i=0.05$ ): one city flagged, but model indicator posterior probability = 0.12 for outlier model vs. 0.88 for basic model — outlier not decisive.
Partial exchangeability (§2.2): Partition $\{\theta_i\}$ into $G$ groups with separate $N(\mu_j,\tau_j^2)$ priors; latent group memberships $g_i \in \{1,\ldots,G\}$ updated Bernoulli: $P(g_i=j|\text{rest}) \propto p_{ij}\phi(\theta_i;\mu_j,\tau_j^2)$ . Reduces to the basic exchangeable model as group locations $|m_j - m_{j'}|\to 0$ .
Mixture link functions (§2.3): Aranda-Ordaz (1981) family $T_\lambda(\eta_i) = \frac{2}{\lambda}\frac{\eta_i^\lambda - (1-\eta_i)^\lambda}{\eta_i^\lambda + (1-\eta_i)^\lambda}$ ( $\lambda=0$ : logit; $\lambda=1$ : linear). Model indicator $I \in \{1,\ldots,L\}$ for $L$ link candidates sampled Bernoulli: $P(I=j|\text{rest}) \propto p_j\pi_j(\mu,\tau^2)$ . Cancer mortality: link probabilities for $v=-0.5, 0, +0.5$ are $0.18, 0.52, 0.30$ — logit preferred.
Fixed-effects vs. random-effects Bayes factor (BF) (§3): $H$ : $\tau^2=0$ (fixed); $K$ : $\tau^2>0$ (random). Mixed prior $p_0\delta_0 + (1-p_0)\rho(\tau^2)$ ; run MCMC on grid of $\tau^2$ values; BF $= [(1-\hat p_1)/\hat p_1] \cdot [p_0/(1-p_0)]^{-1}$ where $\hat p_1 = P(\tau^2>0|y)$ . Validated against Naylor-Smith (1982) adaptive quadrature. Cancer mortality (Table 1): BF peaks at 32.4 ( $\tau=0.202$ ), log BF = 1.51 → positive evidence for random effects.
Math placement (§4.2): 32 items, 200 students. Partial exchangeable with $G=2$ (easy/hard); 28/32 questions classified near 0 or 1; $(m_1,\gamma_1)=(-1,.01)$ , $(m_2,\gamma_2)=(1.4,.01)$ , $(a_i,b_i)=(10,1)$ . Posterior standard deviations (SDs) smaller than classical standard errors (SEs) for most questions; larger SDs for questions at the boundary between groups.
The mixture-model perturbation approach requires only a minimal elaboration of the standard CIHM MCMC — adding one discrete Bernoulli block — making it straightforward to apply to any model sharing the three-stage hierarchical structure.

Concepts Introduced or Extended

Entities Mentioned

Quotes

"Bayes factors can be used to judge the suitability of particular assumptions in CIHMs, including the choice of link function, the nonexistence or existence of outliers, and the prior belief in exchangeability."

"One attractive feature of the proposed mixture model perturbation approach is that it requires straightforward elaborations of standard MCMC methods for fitting exchangeable models."

My Take

This is the cleanest treatment of Bayesian robustness for hierarchical generalized linear models (GLMs) in the early MCMC era. The key insight — that every modeling assumption can be embedded in a discrete mixture and that mixture indicator draws add essentially no cost to the Gibbs sampler — is elegant and broadly applicable. The cancer mortality application is particularly valuable because it shows all three diagnostics on the same dataset: random effects are supported (BF=32), one outlier exists but doesn't meaningfully improve fit (posterior prob 0.12), and the logit link is adequate. The fixed/random effects BF computation via MCMC grid frequencies is a clever pre-cursor to marginal likelihood methods, though the Chib (1995) identity would handle this more directly for non-gridded problems.