The mixed-effects multinomial logistic regression model extends the standard multinomial logit to clustered or longitudinal data by adding subject-specific random effects. For the jth observation within level-2 unit (subject/cluster) i and response category c, the conditional probability is pijc=exp(zijc)/∑h=1Cexp(zijh), where zijc=wij′αc+xij′βic, αc is the fixed-effect vector for contrast c, and βic=Tcθi with θi∼N(0,I) and Σc=TcTc′ (Cholesky). Unlike the Gaussian Mixed-Effects Regression, the marginal likelihood has no closed form and must be approximated numerically.
Key Ideas
Category-specific random effects: Each of the C−1 category contrasts has its own random-effect covariance Σc, so the degree of between-subject heterogeneity can differ across response comparisons.
Flexible contrast coding: The model accommodates any (C−1)×C contrast matrix D in the reparameterized form zijc=wij′Γdc+(xij′⊗θi′)Jr∗Λdc. Reference-cell (one baseline category), Helmert (sequential comparisons), and deviation contrasts are all special cases; the choice should match the substantive research question.
IIA assumption: The multinomial logit embeds independence of irrelevant alternatives — the log-odds for any two categories depends only on those two categories, not on the others. This assumption is reasonable when categories are qualitatively distinct but fails when categories are close substitutes. The Multinomial Probit avoids IIA via a correlated-utility covariance structure.
Category-specific intraclass correlation (ICC): Under a random-intercepts model, the level-1 logistic residual variance is π2/3 (variance of the standard logistic distribution), giving rc=σ^c2/(σ^c2+π2/3) for each contrast c. This allows cluster influence to vary across category comparisons.
MAR missing data: Full likelihood estimation treats missing observations as missing at random (MAR); no imputation is needed for subjects with incomplete follow-up, analogous to the Mixed-Effects Regression case.
How It Works
Conditional likelihood: Given random effects θi, observations within subject i are conditionally independent multinomial draws. The product over time points j and indicator vector yijc∈{0,1} gives ℓ(yi∣θi)=∏j∏cpijcyijc.
Marginal likelihood: h(yi)=∫ℓ(yi∣θ)g(θ)dθ, where g(θ)=N(0,I). This integral has no closed form (contrast with the Gaussian mixed regression model (MRM)); Gauss-Hermite quadrature with Qr points approximates it, where r is the number of random effects per contrast.
Fisher scoring: Gradient and information matrix with respect to (Γ,Λ) are computed by summing over subjects and quadrature points; parameters are updated iteratively until convergence.
EAP estimates: Empirical Bayes (EAP) subject-specific random-effect estimates θ~i=E(θi∣yi) are recovered via one additional quadrature pass after convergence.
Marginal probabilities: Subject-specific probabilities p^ijcss are integrated over g(θ) to obtain population-averaged (marginal) probabilities; for random-intercept models an analytic approximation divides Γ^ by (163/15π)2σ^c2+1.
Quadrature scaling: Standard Gauss-Hermite requires Qr evaluations; for r>5 or so, adaptive quadrature (Liu-Pierce 1994; Pinheiro-Bates 1995) or Markov Chain Monte Carlo (MCMC) via the Gibbs Sampler is preferred.
Why It Matters
Nominal outcomes: Many outcomes of practical interest (type of service use, residential status, voting choice, product category) are unordered categorical; the multinomial logit is the natural model, and mixed effects correct for within-subject correlation.
Contrast flexibility: Helmert and other structured contrasts let the analyst encode scientific hypotheses directly (e.g., "does treatment move people off the streets?" separately from "does treatment move people toward independent vs. community housing?") rather than relying on reference-cell coefficients that may conflate distinct effects.
Intraclass correlations by category: Allowing ICC to vary across contrasts reveals differential clustering — e.g., subjects may cluster more strongly for one type of comparison than another.
Bridge to GLMM: The multinomial logit is an exponential-family model conditional on the random effects; the framework is a special case of the Generalized Linear Mixed Model (GLMM; non-canonical link, vector-valued response).
Open Questions
IIA robustness: When nominal categories are substitutable, multinomial probit or mixed logit (Revelt-Train 1998) with correlated random utilities avoids IIA; mixed-effects extensions of these models exist but are more computationally demanding.
High-dimensional random effects (RE): Quadrature is infeasible for r≫5; adaptive quadrature reduces point counts but adds implementation complexity; full Bayesian MCMC (Gibbs or MH-within-Gibbs) is the standard alternative.
Three-level extensions: Clustered longitudinal data (subjects nested in clinics with repeated measures) require a three-level model; Hedeker (2003) noted this was "underway" at time of publication.
Non-normal RE: Uniform RE distribution produced nearly identical fixed-effects estimates in the MHRP example, suggesting robustness; nonparametric or mixture-of-normals priors offer formal alternatives.
IRT connection: The nominal mixed-effects model with item-specific discrimination parameters (heterogeneous T=[a1⋯am]′) is equivalent to Bock's (1972) nominal item response theory (IRT) model; the reference-cell and Helmert-contrast parameterisations are both special cases of the general contrast framework. See Multilevel Ordinal Regression for the ordinal / Samejima parallel.
Heterogeneous variance: Including zij as a group-indicator vector in the random-effects design yields group-specific SDs, allowing between-cluster variance to differ across subpopulations.