Gelman (2006) Multilevel (Hierarchical) Modeling: What It Can and Cannot Do

Summary

A short methodological tutorial on multilevel (hierarchical) regression, illustrated via the Minnesota radon dataset (919 houses across 85 counties). The paper argues that multilevel modeling excels at prediction by partial pooling — shrinking county-level estimates toward the grand mean while allowing between-county variation — but cautions against causal interpretation of the contextual effect, which conflates individual-level coefficients with unobserved group-level confounders.

Key Claims

• Partial pooling is the core contribution. Complete pooling ($\hat\alpha_j = \hat\alpha$, one intercept for all counties) ignores between-county variation; no pooling ($\hat\alpha_j$ estimated independently per county) overfits small counties. The multilevel estimate $\hat\alpha_j \approx \hat\gamma_0 + \hat\gamma_1 u_j + \text{shrinkage toward } \hat\gamma_0 + \hat\gamma_1 u_j$ is a data-weighted compromise, with more shrinkage for counties with fewer observations.
• Cross-validation validates the claim. Removing individual data points or entire counties and comparing predictions: multilevel estimates outperform both complete and no-pooling at both the individual level and the county level. This is the paper's empirical case for "what it can do."
• The "cannot do" — contextual effects. When an individual-level predictor (e.g., basement indicator $x_{ij}$) also has a group-level average ($\bar x_j$), the model can include both. The coefficient on $\bar x_j$ (the "contextual effect") is often interpreted as the additional causal effect of living in a county with more basements. Gelman argues this interpretation is wrong: $\bar x_j$ is a proxy for unobserved county-level confounders, and the contextual coefficient reflects their aggregate effect, not a causal one.
• Group-level predictors. Adding county-level soil uranium $u_j$ to the county intercept model reduces $\hat\sigma_\alpha^2$ (between-county residual variance), providing a group-level partial $R^2$ analogous to individual-level $R^2$.
• Prior on $\sigma_\alpha$. With $J = 85$ counties, a uniform prior on $(\gamma, \sigma_y, \sigma_\alpha)$ is defensible; with fewer groups, an informative prior on $\sigma_\alpha$ (e.g., half-Cauchy) becomes important.

Concepts Introduced or Extended

• Bayesian Hierarchical Model

Entities Mentioned

• Andrew Gelman

Quotes

"Multilevel modeling is highly effective for predictions at both levels of the model, but could easily be misinterpreted for causal inference."

My Take

An excellent one-page conceptual distillation of what partial pooling achieves and why contextual effects are not causal. The radon example is Gelman's standard pedagogical workhorse; the cross-validation demonstration is clean and convincing. The causal inference caveat is important but underdeveloped in 4 pages — the argument would benefit from a more formal treatment of what identification would require. Paired with Gelman-Pardoe (2004) on explained variance, this forms a self-contained tutorial on hierarchical models.