On MCMC Sampling in Hierarchical Longitudinal Models

Summary

Proposes partially and fully blocked Markov chain Monte Carlo (MCMC) algorithms for Gaussian linear mixed models (LMMs) and binary longitudinal probit models, dramatically reducing the serial autocorrelation that plagues standard one-at-a-time Gibbs samplers. For the Gaussian case, fixed effects $\beta$ can be drawn marginalized over random effects in closed form; the Chib (1995) identity evaluates the integrated likelihood $f(y|\sigma^2,D)$ as a ratio of three Normal densities, enabling a single-block Metropolis-Hastings (M-H) update for all variance parameters. For binary longitudinal data, data augmentation connects to the Chib-Greenberg (1998) multivariate probit; full blocking via the Geweke-Hajivassiliou-Keane (GHK) likelihood evaluation serves as a benchmark. Three real-data examples quantify the autocorrelation gains.

Key Claims

• Standard 4-block Gibbs (Gelfand-Smith 1990, Algorithm 1) for the Gaussian LMM $y_i = X_i\beta + W_i b_i + \varepsilon_i$ mixes slowly when both variance components $D$ and $\sigma^2$ are vague, since the data can only inform them jointly.
• Algorithm 2 (partial block): marginalize $y_i$ over $b_i$ → $y_i|\beta,\sigma^2,D\sim\mathcal{N}_{n_i}(X_i\beta, V_i)$ with $V_i=\sigma^2 I + W_i D W_i'$; draw $\beta|y,\sigma^2,D\sim\mathcal{N}_p(\hat\beta, B)$ without conditioning on $\{b_i\}$; reduces autocorrelation time $\kappa(\beta_1)$ from 20.4 → 1.0 (effectively iid) in the HIV CD4 example.
• Algorithm 3 (single block): use the Chib (1995) identity $f(y|\sigma^2,D)=\phi_p(\beta^*|\beta_0,B_0)\prod_i\phi_{n_i}(y_i|X_i\beta^*,V_i)\,/\,\phi_p(\beta^*|\hat\beta,B)$ (eqs. 2–3, evaluated at posterior mean $\beta^*$) to compute the integrated likelihood; update $\theta=(\sigma^2,\psi)$ jointly by M-H with multivariate-t proposal calibrated on a short Algorithm 2 run; reduces $\kappa(D_{21})$ from 60.1 → 10.9 in the HIV example.
• Extension to Student-t errors: augment with scale factors $\lambda_{ij}\sim\mathcal{G}(\nu/2,\nu/2)$; Algorithm 3 M-H update for $(\sigma^2,\psi|\lambda)$ retains the same structure.
• For binary longitudinal probit: data augmentation $z_{it}|b_i\sim\mathcal{N}(x_{it}'\beta+w_{it}'b_i,1)$; marginalizing $Z_i$ over $b_i$ gives $Z_i\sim\mathcal{N}_{n_i}(X_i\beta, I_{n_i}+W_iDW_i')$, which is a special case of the Chib-Greenberg (1998) multivariate probit (Algorithm 5).
• Algorithm 7 (fully blocked binary): compute $f(y_i|\beta,D)$ via GHK — Cholesky factor $V_i=LL'$, change variable to $\varepsilon_i$, sequentially draw truncated-Normal $\varepsilon_{it}$ and compute product of Normal cumulative distribution functions (CDFs) (eq. 8); average over many GHK draws; update $(\beta,D)$ by M-H; lag-10 autocorrelation for $\beta_2$ drops from 0.934 (Algorithm 4) to 0.009 (Panel Study of Income Dynamics, PSID, labor-force data, two random effects, REs).
• General rule: fixed effects should always be sampled after marginalizing out random effects. More blocking requires more analytic setup but yields dramatically less autocorrelated output.
• For binary data, Algorithm 5 (partial blocking) is the best practical choice; Algorithm 7 is a theoretical benchmark.
• Three empirical applications: (1) HIV ddI/ddC CD4 trial ($N=467$, continuous, 3 REs per patient, $p=9$); (2) Six Cities wheezing study ($I=537$ children, $J=4$ timepoints, binary, single RE); (3) PSID labor-force participation ($N=520$ households, binary, 2 REs).

Concepts Introduced or Extended

• Gibbs Sampler
• Bayesian Hierarchical Model
• Binary Probit
• Marginal Data Density

Entities Mentioned

• Siddhartha Chib
• Bradley P. Carlin

Quotes

"...the parameters of the model may be updated in a single block, greatly improving convergence and producing essentially independent draws from the posterior distribution of interest."

My Take

The canonical reference for blocked Gibbs in hierarchical longitudinal models. The central trick — using the Chib (1995) identity to evaluate the integrated likelihood as a ratio of three Normal density evaluations — is elegant and immediately implementable. Algorithm 2 is the practical workhorse: it eliminates the dominant β-mixing problem at minimal coding cost. Algorithm 3 is ideal when D also mixes poorly; Algorithm 7 for binary data is a benchmark rather than a routine recommendation. The paper explicitly bridges longitudinal and state-space blocking (Carter-Kohn, Chib 1996, Kim-Shephard-Chib 1998), clarifying that the same marginalisation principle governs all three settings.