Liu (2004) Robit Regression: A Simple Robust Alternative to Logistic and Probit Regression

Summary

Liu (2006) proposes robit regression, which replaces the normal cumulative distribution function (CDF) link in probit with a Student-t CDF with $\nu$ degrees of freedom. The robit maximum likelihood estimator (MLE) has a bounded influence function (limit $-\nu$ as $x'\hat\beta\to-\infty$), making it robust to outliers, whereas logistic influence is unbounded and probit influence diverges quadratically. Robit(7) approximates the logistic link to within 0.0006 maximum deviation. Efficient expectation-maximization (EM)-type algorithms (EM, ECME, PX-EM) and accelerated data augmentation (DA) algorithms (E-DA 1, E-DA 2) for Bayesian inference are derived, with the accelerated DA schemes substantially outperforming basic DA via parameter expansion over latent scale and weight variables.

Key Claims

• Bounded influence function: The score factor $f_\nu(x'\hat\beta)/F_\nu(x'\hat\beta)\cdot x'\hat\beta\to-\nu$ as $x'\hat\beta\to-\infty$ for robit. Logistic influence is unbounded; probit diverges as $(x'\hat\beta)^2$. Bounded influence implies no single observation can dominate the MLE.
• Robit(7) ≈ logistic: The $t_7$ CDF with scale $\sigma=1.5484$ matches the logistic CDF to within 0.0006 (maximum distance over the 0.001–0.999 probability range). As $\nu\to\infty$, robit recovers probit exactly.
• Complete-data model: $\tau_i|\theta\sim\text{Gamma}(\nu/2,\nu/2)$, $z_i|(\tau_i,\theta)\sim N(x_i'\beta,1/\tau_i)$, $y_i=\mathbf{1}[z_i>0]$. The E-step computes $\hat\tau_i=E[\tau_i|Y_\text{obs},\theta]$ and $E[\tau_i z_i|Y_\text{obs},\theta]$ analytically as ratios of t-CDF tail probabilities (eqs. 7–9); M-step is weighted least squares (WLS).
• Three MLE algorithms: (1) EM (known $\nu$); (2) ECME (expectation/conditional-maximization either; unknown $\nu$ via CM-step 2 profile-likelihood search over $\nu$ by half-interval method); (3) PX-EM (parameter-expanded EM; Liu-Rubin-Wu 1998 parameter expansion via expanded model $\tau_i/\alpha$, $z_i/\sigma$ — dramatically faster convergence).
• DA for Bayesian inference: Prior $\beta\sim t_p(0,S_0^{-1},\nu_0)$ represented via hierarchical Gamma mixing weight $\tau_0$. I-step draws $z_i$ from truncated $t(\mu_i,1,\nu)$, then $\tau_i\sim\text{Gamma}((\nu+1)/2,\;(\nu+(z_i-\mu_i)^2)/2)$. P-step draws $\beta\sim N(\hat\beta,(\tau_0S_0+S_{\tau xx})^{-1})$.
• Accelerated DA: E-DA 1 rescales $\hat\beta$ by $\chi_n/\sqrt{\text{weighted SS}}$ (integrates out scale of $z$). E-DA 2 further rescales by $(\sum\nu_i\tau_i/\chi^2_{\nu_0+n\nu})^{1/2}$ (integrates out weight scale too) — fastest, as it implicitly integrates out both latent scales.
• Application (Finney 1947): $n=39$ vaso-constriction observations; ECME gives $\hat\nu\approx0.11$ (near-Cauchy, driven by quasi-separation); observations 4, 18, 24 flagged as influential via small fitted weights $\hat\tau_i$, matching Pregibon (1982) resistant fits.

Concepts Introduced or Extended

• Robit Regression
• Data Augmentation
• Binary Probit
• EM Algorithm
• Heavy-Tailed Distributions

Entities Mentioned

• Chuanhai Liu

Quotes

"This paper considers a robit regression, which replaces the normal distribution in probit regression with a t-distribution with a known or unknown number of degrees of freedom."

"The DA algorithms for robit regression model are much simpler to implement than the existing Gibbs sampler for the logistic regression model."

My Take

The robustness argument is clean and the bounded-influence property follows immediately from the heavier t-tails, requiring minimal analytical machinery. The PX-EM acceleration is a high-value addition that exploits the Gamma scale mixture structure for faster MLE convergence at near-zero extra computation. The most important comparison is with Holmes-Held (2006): both papers address binary regression via scale mixtures and DA, but Liu targets ML robustness (t-link with moderate $\nu$) while Holmes-Held targets exact logistic posterior inference (Kolmogorov-Smirnov, KS, mixture). Liu's DA algorithms are simpler — standard Gamma and truncated-t draws only, no generalized inverse Gaussian (GIG) rejection sampler — making them easier to implement in multivariate or high-dimensional extensions. The robit(7) approximation to logistic is very tight (max error 0.0006) but not exact; for strict logistic inference Holmes-Held is preferable.