Summary
Proposes a practical default prior for Bayesian logistic regression: standardize all non-binary predictors to mean 0, standard deviation (SD) 0.5, then place independent Cauchy(0, 2.5) priors on all coefficients and Cauchy(0, 10) on the intercept. Cross-validation on a corpus of applied datasets shows this Cauchy default outperforms maximum likelihood estimation (MLE), flat Gaussian, and Jeffreys priors on predictive accuracy; it also regularizes separation without removing predictors.
Key Claims
- Scaling step. Divide all continuous predictors by 2σ^x (so each has SD 0.5), leaving binary predictors on the 0/1 scale. After this, all predictors are commensurable and the same prior scale applies to each coefficient. Binary and continuous predictors are thus directly comparable.
- Recommended prior. Independent Cauchy(0,2.5) on each scaled coefficient; Cauchy(0,10) on the intercept. This is Student-t with ν=1 degree of freedom (df), center 0, scale 2.5. After scaling, a coefficient of 5 shifts a probability from near 0.5 to near 0.99 — an extreme effect. The Cauchy(0,2.5) says such effects are possible but not the default.
- Why Cauchy over Gaussian. The heavy tails allow large effects when the data support them while still shrinking small effects; a Gaussian prior over-shrinks large genuine signals. This is the key improvement over Raftery's (1996) scaled Gaussian.
- Separation handling. When a predictor perfectly predicts the outcome, MLE diverges to ±∞; the Cauchy prior keeps the posterior finite without removing the predictor. Logistic regression software implementing this prior (the
bayesglm function in R) never crashes due to separation.
- Cross-validation evidence. Evaluated on a corpus of binary regression datasets from published social-science applications: Cauchy(0,2.5) achieves the best predictive log-score among flat, Jeffreys, Gaussian, and Cauchy families. Jeffreys prior (Firth 1993) performs comparably but is harder to extend and interpret as prior information.
- Generalisation. The same two-step procedure (standardize → place t prior) applies to probit, Poisson log-linear, and other generalized linear models (GLMs). The Cauchy is preferred for logistic regression because the logistic distribution itself has heavier tails than the normal; matching tail behavior improves robustness.
- Connection to existing literature. Generalises Raftery (1996) scaled Gaussian to the t family. Related to Zellner's g-prior in that it provides a default scale, but avoids g-prior's dependence on the design matrix. Complements Holmes-Held (2006) which focuses on the Markov chain Monte Carlo (MCMC) algorithm rather than the prior choice.
Concepts Introduced or Extended
Entities Mentioned
Quotes
"We recommend the Cauchy distribution with center 0 and scale 2.5, which in the simplest setting is a longer-tailed version of the distribution attained by assuming one-half additional success and one-half additional failure in a logistic regression."
My Take
This paper solved a practical problem that had lingered in applied Bayesian statistics: what prior to use when you have no specific prior knowledge but MLE is failing due to separation or sparsity. The Cauchy(0, 2.5) default became the standard recommendation in the arm and rstanarm R packages. The scaling step is often overlooked but is essential — without it, the same prior scale applies to a variable measured in millimeters and one measured in kilometers, which is absurd. The cross-validation evidence is convincing but evaluated on a small-to-moderate corpus; the result is robust enough in practice to have withstood widespread adoption. The paper's main limitation is that it addresses classical (non-hierarchical) logistic regression; for hierarchical models, the prior on coefficients is already regularized by the group structure and different considerations apply.