Random Effects Probit

Definition

The random effects probit (REPM) models a single binary outcome for subjects nested within clusters by positing a continuous latent response $y_{ijk}$ for subject $j$ in cluster $i$ on occasion $k$, with response $y_{ijk}=1$ whenever $y_{ijk}$ exceeds a threshold. Correlation within subjects (across occasions) and within clusters (across subjects) is captured by random effects drawn from a normal distribution — in contrast to Generalized Estimating Equations (GEE), which is semiparametric and marginalizes over between-cluster variability.

Key Ideas

• Two-level model (Gibbons-Bock 1987; Gibbons-Hedeker 1994): $y_{ijk} = \beta_{0i} + \mathbf{x}'_{ijk}\boldsymbol{\beta}_{ij} + \mathbf{w}'_{ijk}\boldsymbol{\alpha} + \varepsilon_{ijk}$ with subject random effects $\boldsymbol{\beta}_{ij} \sim N(0, \boldsymbol{\Sigma}_{(2)})$ and residual $\varepsilon_{ijk} \sim N(0, 1)$ (identifiability constraint). Level two covers subjects (repeated measurements within subject) or subjects within cluster.
• Three-level model (Gibbons-Hedeker 1997): adds a cluster-level (level-three) random effect $\beta_{0i} \sim N(0, \sigma^2_{(3)})$; three levels are: measurement occasions (1), subjects (2), clusters (3).
• Conditional independence: given the cluster-level effect $\theta_{(3)}$, subjects within cluster $i$ are independent → the marginal probability factorizes into nested integrals of dimension $r+1$ rather than $(n_i \times r)+1$, making numerical integration tractable.
• Response probability: $P(y_{ijk}=1|\boldsymbol{\beta}^*,\boldsymbol{\alpha}) = \Phi(z_{ijk})$ where $z_{ijk} = \beta_{0i} + \mathbf{x}'_{ijk}\boldsymbol{\beta}_{ij} + \mathbf{w}'_{ijk}\boldsymbol{\alpha}$.
• Logistic variant: replace $\Phi(\cdot)$ with $\Psi(z) = 1/(1+e^{-z})$ and residual variance $1 \to \pi^2/3$; conclusions virtually identical to probit; fixed-effect marginal maximum likelihood estimates (MMLEs) ~40% larger (scale).
• Ordinal extension (Hedeker-Gibbons 1994): for $C$-category ordered outcome, add $C-1$ threshold values; same Gibbs/MMLE machinery.

How It Works

1. Cholesky orthogonalization: reparameterize $\boldsymbol{\beta}^* = \mathbf{T}^*\boldsymbol{\theta}^*$ where $\mathbf{T}^*\mathbf{T}^{*\prime}=\boldsymbol{\Sigma}_{\beta^*}$ (Cholesky factor); standardized $\boldsymbol{\theta}^*$ has identity covariance → standard normal prior for Gauss-Hermite quadrature.
2. Three-level marginal probability (Gibbons-Hedeker 1997, eq. 8): $$h(\mathbf{y}_i) = \int_{\theta_{(3)}} \left\{ \prod_{j=1}^{n_i} \int_{\boldsymbol{\theta}_{(2)}} \left( \prod_{k=1}^{n_{ij}} [\Phi(z_{ijk})]^{1-y_{ijk}} [1-\Phi(z_{ijk})]^{y_{ijk}} \right) g(\boldsymbol{\theta}_{(2)}) d\boldsymbol{\theta}_{(2)} \right\} g(\theta_{(3)}) d\theta_{(3)}$$
3. Fisher scoring: differentiate log likelihood $\sum_i \log h(\mathbf{y}_i)$ with respect to $\boldsymbol{\alpha}$ and unique elements of $\mathbf{T}$; score equations involve nested Gauss-Hermite quadrature.
4. Gaussian factorization (Gibbons-Hedeker 2000): $\boldsymbol{\Sigma}_{\beta^*} = \boldsymbol{\Lambda}\mathbf{D}\boldsymbol{\Lambda}'$ with lower-triangular $\boldsymbol{\Lambda}$ and exponential-transformed diagonal $\mathbf{D}$; preferred over Cholesky when variance components approach zero — avoids convergence failures in MIXOR.
5. Empirical prior (Bock-Aitkin 1981): nonparametric estimate of $g(\boldsymbol{\theta})$ available as robustness check.
6. Intraclass correlation coefficients (ICC): $\text{ICC}_\text{cluster} = \sigma^2_{(3)}/(\sigma^2_{(3)}+\sigma^2_{(2)}+1)$; $\text{ICC}_\text{subject} = \sigma^2_{(2)}/(\sigma^2_{(3)}+\sigma^2_{(2)}+1)$.

Why It Matters

• Ignoring clustering inflates precision: standard errors from a fixed-effect probit are too small when observations are correlated within subjects or clusters, producing spuriously significant results.
• Two-level is often insufficient: multicenter longitudinal trials have three levels (occasion, subject, site); ignoring any one level gives inconsistent estimates and invalid tests.
• Full likelihood enables exact inference: unlike Generalized Estimating Equations, the REPM estimates the full likelihood, enabling Bayes factor model comparison and ICC estimation.
• Distinct from Multivariate Probit: multivariate probit models $p$ simultaneous binary outcomes (health service types); random effects probit models one binary outcome repeated over time and/or nested in clusters. The two are complementary, not competing.
• Computationally tractable: typical practice (1 cluster-level + 1–2 subject-level random effects) requires only $r+1=2$–3 quadrature dimensions.

Open Questions

• Scalability beyond 5–6 random effects requires Gibbs sampling (Geman-Geman 1984) or Markov Chain Monte Carlo (MCMC) data augmentation (Albert-Chib 1993b; Chib-Greenberg 1998).
• Choice of number of random effects at each level: deviance testing may be sample-size sensitive; information criteria may be preferable.
• Non-normal random-effects distributions: the empirical prior (Bock-Aitkin 1981) and rectangular prior produce identical conclusions in the Television School and Family Smoking Prevention study (TVSFP), but formal robustness theory is limited.

Related

• Binary Probit
• Multivariate Probit
• Generalized Estimating Equations
• Bayesian Hierarchical Model
• Marginal Data Density
• Robert D. Gibbons
• Donald Hedeker
• Pattern-Mixture Model
• Mixed-Effects Regression
• Gibbons-Hedeker (1997) Random Effects Probit and Logistic Regression Models for Three-Level Data
• Gibbons-Hedeker (2000) Applications of Mixed-Effects Models in Biostatistics