Heavy-Tailed Distributions

Definition

A distribution is heavy-tailed if its tails decay more slowly than any exponential, meaning extreme values occur with non-negligible probability relative to a normal baseline. In econometrics, the multivariate Student- $t$ distribution — which nests the Cauchy ( $\nu_0 = 1$ ) and converges to the normal as $\nu_0 \to \infty$ — is the canonical heavy-tailed family for regression error modeling, motivated by financial returns, cross-sectional outliers, and instrument-variability settings.

Key Ideas

The multivariate Student- $t$ probability density function (pdf) for an $n$ -vector $u$ with zero mean, scalar dispersion $\sigma^2 I_n$ , and degrees-of-freedom $\nu_0$ is: $p(u|\nu_0, \sigma) \propto \{\nu_0 + u'u/\sigma^2\}^{-(n+\nu_0)/2}$ . Marginal distributions of each $u_i$ are univariate Student- $t$ with $\nu_0$ df.
Elements of $u$ are uncorrelated but not independent — the dependence is captured by a shared latent scale, unlike the multivariate normal where uncorrelated implies independent.
The Student- $t$ is a continuous scale mixture of normals: integrate out $\tau \sim \text{inverted-gamma (IG)}(\nu_0/2, \nu_0\sigma^2/2)$ to get $u \sim \int \mathcal{N}(0, \tau^2 I_n)\, dP(\tau)$ . This representation is the foundation for data-augmented Markov Chain Monte Carlo (MCMC).
The class of scale mixtures of normals is broader than Student- $t$ and includes Laplace, slash, and stable distributions with index $< 2$ — all with heavier tails than the normal.
Small $\nu_0$ produces heavy tails with high kurtosis and slowly decaying tail probabilities; as $\nu_0$ increases, the distribution approaches normality.

How It Works

OLS Robustness Under Student-t Errors (Zellner 1976)

For $y = X\beta + u$ with $u \sim \text{Student-}t(\nu_0, 0, \sigma^2 I_n)$ :

ordinary least squares (OLS) = maximum likelihood estimator (MLE): $\hat\beta = (X'X)^{-1}X'y$ maximizes the likelihood for any monotonically decreasing function of $u'u$ , which the Student- $t$ likelihood satisfies. So OLS remains the MLE regardless of $\nu_0$ .
OLS = minimum variance linear unbiased estimator (MVLUE): When $\nu_0 > 2$ , the Gauss-Markov conditions hold (zero mean, finite variance, uncorrelated errors), so $\hat\beta$ is also the MVLUE.
$t$ / $F$ statistics valid: Given the mixing variable $\tau$ , $u|\tau \sim \mathcal{N}(0, \tau^2 I_n)$ , so $t$ and $F$ statistics are standard. The IG mixing pdf is proper and independent of $t$ and $F$ , preserving their marginal distributions.
Scale inference changes: $s^2/\sigma^2 \sim F(\nu, \nu_0)$ , not $\chi^2(\nu)/\nu$ . Tests and intervals for $\sigma^2$ require the $F$ distribution; the departure from $\chi^2$ shrinks as $\nu_0 \to \infty$ .
$\nu_0$ not jointly ML-estimable: No maximum exists over $(\beta, \sigma^2, \nu_0)$ simultaneously; $\nu_0$ must be treated as known or given a proper prior.

Bayesian Analysis with Diffuse Prior (Zellner 1976)

Under $p(\beta, \sigma^2) \propto 1/\sigma^2$ :

Marginal posterior for $\beta$ : $k$ -dimensional Student- $t$ , identical in form to the normal-model posterior. $\beta$ inference is invariant to $\nu_0$ — departures from normality do not affect regression coefficient conclusions.
Marginal posterior for $\sigma^2/s^2$ : $F(\nu_0, \nu)$ . Converges to $\chi^2(\nu)/\nu$ as $\nu_0 \to \infty$ .
Natural conjugate prior: $p_F(\sigma^2|\cdot) \times p_S(\beta|\sigma^2, \cdot)$ — an $F$ pdf for $\sigma^2$ times a conditional multivariate Student- $t$ for $\beta$ ; collapses to normal-gamma conjugate as $\nu_0 \to \infty$ .

Independent Student-t Errors (Geweke 1993)

Geweke (1993) treats the case where errors are independent (not jointly multivariate) Student-t, each with individual scale mixing variable $\lambda_i \sim \mathrm{IG}(\nu_0/2, \nu_0/2)$ . Augmenting the model with $\lambda_1, \ldots, \lambda_n$ and $\nu_0$ yields a data-augmented Gibbs sampler with conjugate full conditionals, generalising the Zellner framework to allow estimation of $\nu_0$ .

Why It Matters

Financial returns exhibit excess kurtosis far above 3; assuming normality understates tail risk and distorts inference.
Cross-sectional data (firms, individuals) frequently contain outlier observations that inflate OLS variance under normal assumptions.
Regression robustness: Zellner (1976) shows precisely where heavy tails matter — $\beta$ inference survives, $\sigma^2$ inference does not — guiding practitioners on which adjustments are necessary.
MCMC tractability: The scale-mixture representation converts an intractable non-normal likelihood into a sequence of conjugate conditional posteriors, enabling efficient Gibbs sampling by augmenting the model with latent scale variables $\tau$ (or individual $\lambda_i$ ).
Motivating examples: (a) measuring instruments whose run-to-run variability $\tau$ is unknown but random; (b) stock return regressions where a common random standard deviation across observations is plausible; (c) cross-group regressions where groups share a common random error variance.

Open Questions

Optimal specification or estimation of $\nu_0$ when it cannot be ML-estimated: assigning a uniform or log-uniform prior over $\nu_0$ and marginalizing remains the standard approach but introduces sensitivity.
Extension to seemingly unrelated regression (SUR) with matrix-variate Student- $t$ errors: Zellner (1976) conjectures similar results hold but leaves the proof to future work.
Robustness of the $\beta$ -invariance result when $\nu_0$ is itself estimated (posterior for $\beta$ no longer integrates out $\nu_0$ cleanly).