Mixed-Effects Regression

Definition

The mixed-effects regression model (MRM) for continuous outcomes decomposes each subject's response trajectory into population-average fixed effects and individual-specific random effects: $\mathbf{y}_i = \mathbf{W}_i\boldsymbol{\alpha} + \mathbf{X}_i\boldsymbol{\beta}_i + \boldsymbol{\varepsilon}_i$, where $\boldsymbol{\alpha}$ are fixed effects common to all subjects, $\boldsymbol{\beta}_i \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma}_\beta)$ are subject-specific random deviations, and $\boldsymbol{\varepsilon}_i \sim \mathcal{N}(\mathbf{0}, \sigma^2_\varepsilon \boldsymbol{\Omega}_i)$ is the within-subject error with autocorrelation structure $\boldsymbol{\Omega}_i$.

Key Ideas

• Two-level structure: Level 1 = repeated measurements within subjects; Level 2 = subjects (optionally nested in clusters for three-level extensions). Fixed effects $\boldsymbol{\alpha}$ describe population trends; random effects $\boldsymbol{\beta}_i$ describe individual deviation from those trends.
• Empirical Bayes (EAP) estimates: Subject-specific random effects are estimated as $\tilde{\boldsymbol{\beta}}_i = [\mathbf{X}_i'(\sigma^2_\varepsilon\boldsymbol{\Omega}_i)^{-1}\mathbf{X}_i + \boldsymbol{\Sigma}_\beta^{-1}]^{-1}\mathbf{X}_i'(\sigma^2_\varepsilon\boldsymbol{\Omega}_i)^{-1}(\mathbf{y}_i - \mathbf{W}_i\boldsymbol{\alpha})$ — a weighted average of the subject's ordinary least squares (OLS) estimate and the population mean, with shrinkage proportional to $\boldsymbol{\Sigma}_\beta^{-1}$.
• Intraclass correlation coefficient (ICC) decomposition: For a two-level design with random intercept $\sigma^2_b$ and residual $\sigma^2_\varepsilon$, $\text{ICC} = \sigma^2_b / (\sigma^2_b + \sigma^2_\varepsilon)$ quantifies the proportion of total variance attributable to between-subject differences.
• Gaussian factorization: $\boldsymbol{\Sigma}_\beta = \boldsymbol{\Lambda}\mathbf{D}\boldsymbol{\Lambda}'$ with lower-triangular $\boldsymbol{\Lambda}$ (ones on diagonal) and diagonal $\mathbf{D}$ with exponential-transformed entries $\exp(d_k)$; avoids convergence failures when variance components near zero compared to Cholesky decomposition.
• Missing data (MAR): Likelihood-based MRM is valid under missing-at-random (MAR) dropout without imputation; the full-data likelihood marginalizes over unobserved responses given observed ones.
• Pattern-mixture extension: Nonignorable, missing-not-at-random (MNAR) dropout handled by including dropout pattern as between-subjects covariate; see Pattern-Mixture Model.

How It Works

1. Marginal likelihood: $h(\mathbf{y}_i) = \int \mathcal{N}(\mathbf{y}_i; \mathbf{W}_i\boldsymbol{\alpha} + \mathbf{X}_i\boldsymbol{\beta}_i, \sigma^2_\varepsilon\boldsymbol{\Omega}_i) \cdot \mathcal{N}(\boldsymbol{\beta}_i; \mathbf{0}, \boldsymbol{\Sigma}_\beta) \, d\boldsymbol{\beta}_i$ has closed form (normal-normal conjugacy): $h(\mathbf{y}_i) = \mathcal{N}(\mathbf{W}_i\boldsymbol{\alpha}, \mathbf{V}_i)$ with $\mathbf{V}_i = \mathbf{X}_i\boldsymbol{\Sigma}_\beta\mathbf{X}_i' + \sigma^2_\varepsilon\boldsymbol{\Omega}_i$.
2. Fisher scoring: Marginal maximum likelihood estimator (MMLE) for $\boldsymbol{\alpha}$, $\boldsymbol{\Sigma}_\beta$, and $\sigma^2_\varepsilon$ via iterative reweighted least squares; closed-form gradient and Hessian available from the normal marginal.
3. Autocorrelation structures: $\boldsymbol{\Omega}_i$ can be autoregressive (AR(1)), banded, or unstructured; MIXREG implements AR(1) as default for equally-spaced measurements.
4. Three-level extension: Cluster-level (therapist/school) random effects added as a third level; conditional independence on the cluster effect reduces the subject-level integral.
5. Software: MIXREG (Hedeker-Gibbons 1996b) for continuous outcomes; free download, command-line interface; superseded in practice by R's lme4 and nlme packages.

Why It Matters

• Efficiency over OLS: Joint modeling of all time points uses the full covariance structure; OLS on subject-level means discards within-subject information.
• Unbalanced designs: Subjects with different observation schedules or missing data contribute their actual observations; no imputation or case deletion required.
• Subject-specific inference: EAP estimates enable individual-level prediction (e.g., therapist performance ranking, school-level policy assessment) with appropriate shrinkage toward the population mean.
• Interaction detection: Fixed-effects analysis of variance (ANOVA) ignores between-cluster clustering, inflating Type I error; the MRM correctly attributes experience × treatment interactions invisible to marginal analyses.
• Foundation for extensions: The continuous MRM framework generalizes to binary outcomes (Random Effects Probit), ordinal outcomes (Hedeker-Gibbons 1994), multivariate outcomes (Multivariate Probit), survival data, and non-Gaussian exponential-family responses via the Generalized Linear Mixed Model.

Open Questions

• Number of random effects: deviance tests are conservative at boundary (variance = 0); information criteria preferred.
• Robustness to non-normal random effects: EAP shrinkage is optimal under normality; robust alternatives (empirical nonparametric prior, Bock-Aitkin 1981) are available but rarely used in practice.
• MNAR vs. MAR: pattern-mixture and selection models give different identifications; sensitivity analysis across dropout models recommended.

Related

• Pattern-Mixture Model
• Random Effects Probit
• Multivariate Probit
• Generalized Estimating Equations
• Generalized Linear Mixed Model
• Bayesian Hierarchical Model
• Random Coefficient Model
• Robert D. Gibbons
• Donald Hedeker
• Gibbons-Hedeker (2000) Applications of Mixed-Effects Models in Biostatistics
• van Dyk (2000) Fitting Mixed-Effects Models Using Efficient EM-Type Algorithms