Fong-Rue-Wakefield (2010) Bayesian Inference for Generalized Linear Mixed Models

Summary

This paper makes Bayesian inference for generalized linear mixed models (GLMMs) practical by using integrated nested Laplace approximations (INLA) instead of MCMC, and by giving concrete guidance on how to specify priors for the hard part — the variance components. GLMMs combine a generalized linear model with normal random effects on the linear-predictor scale; their flexibility costs analytical tractability, so likelihood inference (via penalized quasi-likelihood) can be unreliable in small samples and full Bayesian MCMC is computationally heavy and prior-sensitive. Fong, Rue, and Wakefield rework the classic Breslow-Clayton (1993) examples with INLA, specify priors on interpretable quantities (including the implied degrees of freedom of a spline), and conclude that Bayesian GLMM analysis is now fast and attractive — while warning that all approximation strategies degrade for clustered binary data.

Key Claims

• GLMM = GLM + Gaussian random effects. On the linear-predictor scale $g(\mu_{ij})=x_{ij}'\beta+z_{ij}'b_i$ with $b_i\sim\mathcal N(0,\Sigma)$; the resulting likelihood requires integrating out the random effects, which is analytically intractable and the source of all the computational difficulty.
• INLA replaces MCMC. GLMMs are latent Gaussian models, so the integrated nested Laplace approximation (Rue-Martino-Chopin 2009) applies: it deterministically approximates the marginal posteriors of the hyperparameters and latent field by nested Laplace approximations, giving accurate posteriors in seconds to minutes without Markov-chain sampling or convergence diagnostics.
• Priors on variance components are the crux. Weakly informative or improper priors on random-effect variances can be surprisingly influential and can yield improper posteriors; the paper recommends specifying priors on interpretable scales — e.g., the marginal standard deviation of the random effect, or, for penalized-spline models, the prior on the smoothing variance chosen to match a target implied degrees of freedom.
• Reworks Breslow-Clayton. All the examples of the seminal PQL paper (Breslow-Clayton 1993) are redone in the Bayesian/INLA framework, showing the Bayesian answers and the effect of prior choices side by side with the likelihood results.
• A caution for binary clustered data. As with PQL, the Laplace-type approximations underlying INLA are least accurate for clustered binary responses (little information per cluster), so results there require extra care and checking.

Concepts Introduced or Extended

• Generalized Linear Mixed Model
• Laplace Approximation

Entities Mentioned

• Youyi Fong
• Håvard Rue
• Jon Wakefield

Quotes

"A Bayesian approach is appealing but has been hampered by the lack of a fast implementation, and the difficulty in specifying prior distributions with variance components again being particularly problematic."

"Bayesian inference is now practically feasible for GLMMs and provides an attractive alternative to likelihood-based approaches such as penalized quasi-likelihood."

My Take

The paper's importance is less a new model than a change in what is feasible: by recognizing the GLMM as a latent Gaussian model and handing it to INLA, it removes the two frictions — slow, finicky MCMC and opaque prior choices — that kept applied Bayesian GLMM analysis rare. The prior-specification advice is the underrated contribution: putting the prior on a marginal standard deviation or an implied degrees-of-freedom is exactly the kind of interpretable elicitation the wiki's hierarchical-model and Laplace-approximation pages need, and it is the sensible middle ground between reflexive "non-informative" gamma priors (which are not non-informative for variances) and fully subjective priors. The honest caveat, which the authors foreground, is that deterministic approximations and clustered binary data do not mix well — the one regime where you still want to check against MCMC.