Zellner (1976) Bayesian and Non-Bayesian Analysis of the Regression Model with Multivariate Student-t Error Terms

Summary

Zellner analyzes the linear regression model $y = X\beta + u$ under the assumption that $u$ has a joint multivariate Student-t distribution — a broader class than normality that includes the Cauchy ( $\nu_0 = 1$ ) and normal ( $\nu_0 \to \infty$ ) as special cases, and can be represented as a continuous scale mixture of normals. He shows that the Ordinary Least Squares (OLS) estimator is both the Maximum Likelihood Estimator (MLE) and minimum variance linear unbiased estimator, classical t- and F-statistics remain marginally valid despite non-normal errors, but inference about the scale parameter $\sigma^2$ requires an F-distribution rather than the usual $\chi^2$ . Under a diffuse Bayesian prior, the marginal posterior for $\beta$ takes exactly the same Student-t form as from the normal model and is invariant to $\nu_0$ , while the posterior for $\sigma^2$ follows an F distribution. A natural conjugate prior for the multivariate Student-t regression model is also derived.

Key Claims

The multivariate Student-t probability density function (pdf) for $u$ is a continuous scale mixture of normals: integrate out $\tau \sim \text{IG}(\nu_0/2,\, \nu_0\sigma^2/2)$ to get $u|\tau \sim \mathcal{N}(0, \tau^2 I_n)$ . Elements of $u$ are uncorrelated but not independent.
OLS $\hat\beta = (X'X)^{-1}X'y$ is the MLE for $\beta$ for any likelihood monotonically decreasing in $u'u$ ; also Minimum Variance Linear Unbiased Estimator (MVLUE) when $\nu_0 > 2$ (Gauss-Markov conditions met given uncorrelated, zero-mean, finite-variance errors).
Classical t- and F-statistics are marginally valid: given the mixing variable $\tau$ , $u$ is normal so t and F have standard distributions; since $\tau$ has a proper IG marginal independent of them, marginal t and F distributions are also standard.
For $\sigma^2$ inference, $s^2/\sigma^2 \sim F(\nu, \nu_0)$ rather than $\chi^2(\nu)/\nu$ . Confidence intervals and tests for $\sigma^2$ require the F distribution, not $\chi^2$ .
$\nu_0$ is not jointly ML-estimable with $\beta$ and $\sigma^2$ : the likelihood has no maximum over $(\beta, \sigma^2, \nu_0)$ simultaneously. $\nu_0$ must be fixed or given a proper prior.
Under diffuse prior $p(\beta, \sigma^2) \propto 1/\sigma^2$ , the marginal posterior for $\beta$ is a $k$ -dimensional Student-t — identical in form to the normal-model diffuse-prior posterior. Posterior $\beta$ inference is invariant to $\nu_0$ .
Marginal posterior for $\sigma^2/s^2$ follows $F(\nu_0, \nu)$ . As $\nu_0 \to \infty$ , converges to the $\chi^2(\nu)/\nu$ normal-model result.
Natural conjugate prior: $p_F(\sigma^2|\cdot) \times p_S(\beta|\sigma^2, \cdot)$ — an F pdf for $\sigma^2$ times a conditional multivariate Student-t for $\beta$ . Collapses to the normal-gamma conjugate when $\nu_0 \to \infty$ .

Concepts Introduced or Extended

Entities Mentioned

Arnold Zellner

Quotes

"It is important to observe that even though the elements of u have the nonnormal pdf in (2.2) and are not independent, tests and intervals based on usual t- and F-statistics remain valid."

"The marginal posterior pdf for $\beta$ ...does not depend on the value given to $\nu_0$ . In fact, (3.5) is precisely the result that one obtains in the Bayesian analysis of the normal regression model with the diffuse prior."

My Take

A seminal six-page paper that establishes the multivariate Student-t regression model as theoretically well-grounded and practically tractable. The key insight — that $\beta$ inference is completely robust to departures from normality while $\sigma^2$ inference is not — cleanly partitions the consequences of heavy tails. The scale-mixture-of-normals representation anticipates later Markov Chain Monte Carlo (MCMC) developments in robust Bayesian modeling. The main limitation is that $\nu_0$ cannot be estimated jointly by ML and must be fixed or given a prior, which the paper acknowledges without resolution.