Multilevel Model

Definition

A multilevel model (also called a hierarchical linear model, mixed-effects model, or random-effects model) represents data with a nested structure — units $i$ within groups $j$ — by decomposing variation into within-group and between-group components. The linear predictor takes the form $\eta_{ij} = \alpha + \beta' x_{ij} + \xi_j + \epsilon_{ij}$, where $\xi_j \sim \mathcal{N}(0, \sigma_\xi^2)$ is a group-level random effect and $\epsilon_{ij}$ is residual error. Generalized Linear Mixed Model (GLMM) extensions allow non-Gaussian outcomes via a link function.

Key Ideas

• Partial pooling: Group-level estimates shrink toward the overall mean in proportion to group size and $\sigma_\xi^2$; small groups borrow strength from the full sample. The shrinkage coefficient is $\lambda_j = \sigma_\xi^2/(\sigma_\xi^2 + \sigma_\epsilon^2/n_j)$ — Bühlmann credibility in actuarial terms.
• Intraclass correlation (ICC): $\rho = \sigma_\xi^2/(\sigma_\xi^2 + \sigma_\epsilon^2)$ measures the proportion of total variance attributable to group membership. ICC near 0 → ordinary least squares (OLS) with cluster-robust standard errors (SEs) suffices; ICC near 1 → group effects dominate.
• Random intercepts vs. random slopes: The simplest model allows only intercepts $\xi_j$ to vary across groups. Random slopes extend this to allow coefficient heterogeneity: $\eta_{ij} = \alpha_j + \beta_j' x_{ij}$, with $(\alpha_j, \beta_j)$ jointly normal.
• GLMM extension: Replace the linear response with $g(\mu_{ij}) = \eta_{ij}$ for link $g$. For multinomial logit with $M$ categories: $\eta_{ij}^{(m)} = \alpha^{(m)} + \beta^{(m)'} x_{ij} + \xi_j^{(m)}$, where cluster-level effects $(\xi_j^{(1)},\ldots,\xi_j^{(M-1)})$ are jointly normal with unrestricted covariance matrix.
• Selection extension (S&P model): When outcome is observed only for a selected subset (non-ignorable missing data), a joint selection-and-principal (S&P) equation links a binary selection indicator to the principal multinomial equation via shared cluster random effects. Identification requires individual-level exclusion restrictions. Grilli-Rampichini (2006) show the subject-level selection-principal covariance is unidentified and omit it.
• Estimation: Marginal likelihood via adaptive Gaussian quadrature (Stata gllamm); Laplace approximation (R lme4); full Bayesian Markov Chain Monte Carlo (MCMC) (Stan, JAGS) propagates uncertainty in variance components.

How It Works

1. Specify random-effect distribution: $\xi_j \sim \mathcal{N}(0, \sigma_\xi^2)$ (or multivariate normal for multiple random effects).
2. Integrate out $\xi_j$ to obtain the marginal likelihood $p(y_j|\theta) = \int \prod_i p(y_{ij}|\xi_j,\theta) \, p(\xi_j) \, d\xi_j$.
3. For GLMMs the integral is intractable; evaluate numerically via Gauss-Hermite quadrature (accuracy increases with number of quadrature points; adaptive quadrature recenters nodes at the posterior mode of $\xi_j$).
4. Estimate $(\beta, \sigma_\xi^2)$ by maximizing the marginal log-likelihood.
5. Recover group-level predictions via the conditional mode (best linear unbiased predictors, BLUPs): $\hat\xi_j = \mathrm{E}[\xi_j | y_j, \hat\theta]$.

Why It Matters

• Correct inference: Ignoring clustering inflates effective sample size, leading to anti-conservative tests. Multilevel models give correct standard errors for cluster-correlated data.
• Explanatory decomposition: ICC quantifies how much of the outcome variance is "explained" by group membership — useful for policy (e.g., how much of skill acquisition is a university-level versus individual-level phenomenon).
• Non-ignorable missing data: The S&P extension (Grilli-Rampichini 2006) handles settings where missingness depends on unobserved outcomes, with identification from exclusion restrictions rather than functional-form assumptions.

Open Questions

• Number of groups: Standard asymptotics require the number of groups $J \to \infty$; with small $J$ (e.g., $J < 5$), variance components are poorly identified and Bayesian priors on $\sigma_\xi^2$ matter substantially.
• Cross-classified vs. nested: Multilevel logic extends to cross-classified designs (students in schools × neighborhoods), but estimation is considerably harder.
• Bayesian vs. frequentist shrinkage: Both give partial pooling, but Bayesian credible intervals for variance components are generally wider and better calibrated in small samples.

Related

• Bayesian Hierarchical Model
• Selection Model
• Grilli-Rampichini (2006)
• Leonardo Grilli
• Carla Rampichini