Scale Mixture of Normals

Definition

A scale mixture of normals writes a symmetric, heavier-than-Gaussian density as a continuous mixture of zero-mean normals whose variance is randomised by a latent positive scaling variable $\lambda$ : $p(u) = \int_0^\infty \mathcal{N}(u \mid 0,\, \lambda\sigma^2)\,\pi(\lambda)\,d\lambda$ . Conditional on $\lambda$ the variable is exactly Gaussian; marginally, integrating $\lambda$ out against the mixing density $\pi(\lambda)$ produces excess kurtosis and heavy tails while preserving symmetry. Different choices of $\pi(\lambda)$ recover different heavy-tailed families — Student- $t$ , Laplace, logistic, slash, and symmetric stable laws all arise this way — which makes the representation the unifying device behind conjugate Bayesian computation for robust and heavy-tailed models.

Key Ideas

One latent scale per observation. Each datum gets its own mixing weight $\lambda_i$ : $u_i \mid \lambda_i \sim \mathcal{N}(0, \lambda_i\sigma^2)$ . Large $\lambda_i$ inflates the variance for that observation, letting the model absorb an outlier without distorting the common $\sigma^2$ or the regression coefficients.
Student- $t$ = normal scaled by inverse-gamma $\lambda$ . With $\nu/\lambda_i \sim \chi^2(\nu)$ (equivalently $\lambda_i^{-1} \sim \text{Gamma}(\nu/2, \nu/2)$ ) the marginal is Student- $t_\nu$ ; this is the case treated statically for regression by Geweke (1993) Bayesian Treatment of the Independent Student-t Linear Model and jointly (multivariate- $t$ , shared scale) by Zellner (1976) Bayesian and Non-Bayesian Analysis of the Regression Model with Multivariate Student-t Error Terms.
Laplace / double-exponential = normal scaled by exponential $\lambda$ . An exponential mixing density on the variance yields the double-exponential law — the representation underlying the Bayesian LASSO, where the double-exponential prior on coefficients becomes conditionally Gaussian given the scales.
Logistic $\approx$ normal scaled by Kolmogorov-Smirnov $\lambda$ . Holmes-Held (2006) Bayesian Auxiliary Variable Models for Binary and Multinomial Regression write $\varepsilon_i \sim \text{Logistic}(0,1)$ exactly as $\mathcal{N}(0, \lambda_i)$ with $\lambda_i = (2\psi_i)^2$ , $\psi_i \sim$ Kolmogorov-Smirnov — the exact replacement for the earlier approximate $t_8$ trick (later superseded by the related Pólya-Gamma scheme).
Distinct from a discrete finite mixture. A scale mixture is a continuous mixture over a scalar variance ( $K = \infty$ components sharing mean zero); a finite Mixture of Normals is a convex sum of $K$ fixed components with their own means and weights, used to approximate a fixed target such as the $\log\chi^2$ error in stochastic volatility.

How It Works

The representation converts a non-conjugate, heavy-tailed likelihood into a conditionally Gaussian one, so Data Augmentation restores a simple Gibbs step. Augment the parameter space with the latent scales $\lambda = (\lambda_1,\ldots,\lambda_n)$ and cycle:

Scales given everything else — $\lambda_i \mid u_i$ is a standard draw fixed by the mixing family. For the Student- $t$ case (Geweke 1993, eq. 15), $[(\sigma^{-2}u_i^2 + \nu)/\lambda_i] \sim \chi^2(\nu+1)$ , i.e. an inverse-gamma / gamma update; for the logistic case a generalized-inverse-Gaussian rejection draw.
Coefficients given scales — conditional on $\lambda$ the model is a heteroskedastic Gaussian regression, so $\beta \mid \lambda, \sigma$ is the generalized-least-squares Normal $\hat\beta(\lambda) = (X'\Lambda^{-1}X)^{-1}X'\Lambda^{-1}y$ .
Common scale $\sigma^2 \mid \beta, \lambda$ — conjugate inverse-gamma.
Degrees of freedom $\nu$ (if unknown) — a non-standard one-dimensional draw (Metropolis, grid, or a proper prior such as exponential).

The same augmentation threads through binary and dynamic models. In robit / robust probit (Albert-Chib (1993b) Bayesian Analysis of Binary and Polychotomous Response Data, Robit Regression) the latent utility carries a scale weight $\lambda_i \sim \text{Gamma}(\nu/2, 2/\nu)$ , giving a $t$ -link that downweights outliers; the fitted $\hat\lambda_i \ll 1$ double as an outlier diagnostic. In heavy-tailed Stochastic Volatility the return error is Student- $t$ by scale mixing, $u_t = \lambda_t^{-1/2}\varepsilon_t$ with $\lambda_t \sim \text{Gamma}(\nu/2, \nu/2)$ : Chib-Nardari-Shephard (2002) MCMC Methods for Stochastic Volatility Models combine this with the fixed 7-component finite mixture that approximates $\log\chi^2(1)$ , and Omori-Chib-Shephard-Nakajima (2007) Stochastic Volatility with Leverage: Fast and Efficient Likelihood Inference carry the Student- $t$ scale mixture into the leverage model. Here the two mixture ideas coexist: the scale mixture supplies fat tails, while a discrete finite mixture approximates the log-chi-square measurement noise.

Why It Matters

Conjugacy on demand. Any symmetric heavy-tailed error becomes conditionally Gaussian, so an intractable posterior is reduced to a sequence of conjugate conditionals — the mechanism behind robust regression, the $t$ - and logistic-link binary models, Bayesian LASSO shrinkage, and heavy-tailed SV.
Outlier accommodation with a diagnostic. Per-observation scales let a few extreme points inflate their own variance instead of biasing $\beta$ ; the posterior weights $\hat\lambda_i$ flag which observations were treated as outliers.
Robustness of conclusions. Zellner (1976) shows that under a multivariate- $t$ (shared-scale) mixture the posterior for $\beta$ is invariant to the degrees of freedom, while $\sigma^2$ inference is not — cleanly partitioning where heavy tails do and do not matter.
A template that generalises. Geweke's static cross-sectional weights $\lambda_i$ are the direct precursor of the time-varying latent variance path in SV models; the same augmentation logic scales from a single regression to dynamic state-space models.

Open Questions