Scale Mixture of Normals

scale-mixtureheavy-tailed-distributionsdata-augmentationstudent-trobust-regressionbayesian

Definition

A scale mixture of normals writes a symmetric, heavier-than-Gaussian density as a continuous mixture of zero-mean normals whose variance is randomised by a latent positive scaling variable λ\lambda: p(u)=∫0∞N(u∣0, λσ2) π(λ) dλp(u) = \int_0^\infty \mathcal{N}(u \mid 0,\, \lambda\sigma^2)\,\pi(\lambda)\,d\lambda. Conditional on λ\lambda the variable is exactly Gaussian; marginally, integrating λ\lambda out against the mixing density π(λ)\pi(\lambda) produces excess kurtosis and heavy tails while preserving symmetry. Different choices of π(λ)\pi(\lambda) recover different heavy-tailed families — Student-tt, Laplace, logistic, slash, and symmetric stable laws all arise this way — which makes the representation the unifying device behind conjugate Bayesian computation for robust and heavy-tailed models.

Key Ideas

How It Works

The representation converts a non-conjugate, heavy-tailed likelihood into a conditionally Gaussian one, so Data Augmentation restores a simple Gibbs step. Augment the parameter space with the latent scales λ=(λ1,…,λn)\lambda = (\lambda_1,\ldots,\lambda_n) and cycle:

  1. Scales given everything else — λi∣ui\lambda_i \mid u_i is a standard draw fixed by the mixing family. For the Student-tt case (Geweke 1993, eq. 15), [(σ−2ui2+ν)/λi]∼χ2(ν+1)[(\sigma^{-2}u_i^2 + \nu)/\lambda_i] \sim \chi^2(\nu+1), i.e. an inverse-gamma / gamma update; for the logistic case a generalized-inverse-Gaussian rejection draw.
  2. Coefficients given scales — conditional on λ\lambda the model is a heteroskedastic Gaussian regression, so β∣λ,σ\beta \mid \lambda, \sigma is the generalized-least-squares Normal β^(λ)=(X′Λ−1X)−1X′Λ−1y\hat\beta(\lambda) = (X'\Lambda^{-1}X)^{-1}X'\Lambda^{-1}y.
  3. Common scale σ2∣β,λ\sigma^2 \mid \beta, \lambda — conjugate inverse-gamma.
  4. Degrees of freedom ν\nu (if unknown) — a non-standard one-dimensional draw (Metropolis, grid, or a proper prior such as exponential).

The same augmentation threads through binary and dynamic models. In robit / robust probit (Albert-Chib (1993b) Bayesian Analysis of Binary and Polychotomous Response Data, Robit Regression) the latent utility carries a scale weight λi∼Gamma(ν/2,2/ν)\lambda_i \sim \text{Gamma}(\nu/2, 2/\nu), giving a tt-link that downweights outliers; the fitted λ^i≪1\hat\lambda_i \ll 1 double as an outlier diagnostic. In heavy-tailed Stochastic Volatility the return error is Student-tt by scale mixing, ut=λt−1/2εtu_t = \lambda_t^{-1/2}\varepsilon_t with λt∼Gamma(ν/2,ν/2)\lambda_t \sim \text{Gamma}(\nu/2, \nu/2): Chib-Nardari-Shephard (2002) MCMC Methods for Stochastic Volatility Models combine this with the fixed 7-component finite mixture that approximates log⁡χ2(1)\log\chi^2(1), and Omori-Chib-Shephard-Nakajima (2007) Stochastic Volatility with Leverage: Fast and Efficient Likelihood Inference carry the Student-tt scale mixture into the leverage model. Here the two mixture ideas coexist: the scale mixture supplies fat tails, while a discrete finite mixture approximates the log-chi-square measurement noise.

Why It Matters

Open Questions

Related