Tiao-Zellner (1964) On the Bayesian Estimation of Multivariate Regression

Summary

This paper gives the Bayesian analysis of a system of regression equations with correlated errors — the multivariate-regression / seemingly-unrelated-regressions (SUR) model — under a diffuse prior. For the $m$-equation model $y_\alpha = X_\alpha\beta_\alpha + u_\alpha$ with disturbances jointly normal and cross-equation covariance $\Sigma\otimes I_T$, Tiao and Zellner derive the posterior distributions in closed form. When the regressor matrix is the same in every equation (the classical multivariate multiple-regression case), the marginal posterior of each equation's coefficient vector is multivariate-$t$, and the error covariance matrix $\Sigma$ has an inverse (inverted) Wishart posterior; they also derive and discuss the joint posterior of all the regression coefficients in the general (different-regressors) SUR model. (Journal of the Royal Statistical Society, Series B 26(2): 277–285.)

Key Claims

• The model. An $m$-equation system $y_\alpha = X_\alpha\beta_\alpha + u_\alpha$ ($\alpha=1,\dots,m$), stacked as $y = Z\beta + u$ with $Z$ block-diagonal, where the disturbances are jointly normal with mean zero and covariance $\Sigma\otimes I_T$ ($\Sigma$ an $m\times m$ positive-definite cross-equation covariance). This is the seemingly-unrelated-regressions structure: equations linked only through correlated errors.
• Diffuse-prior Bayesian treatment. Using a non-informative (Jeffreys-type) prior on $(\beta,\Sigma)$, the likelihood $\ell(\beta,\Sigma\mid y)\propto |\Sigma|^{-T/2}\exp\{-\tfrac12 (y-Z\beta)'(\Sigma^{-1}\otimes I_T)(y-Z\beta)\}$ is combined with the prior and the nuisance parameters integrated out analytically.
• Multivariate-$t$ coefficient posteriors. In the special case where $X$ is the same for all equations (traditional multivariate regression), the marginal posterior of any single equation's coefficient vector $\beta_\alpha$ — after integrating out $\Sigma$ — is a multivariate Student-$t$ distribution, the multi-equation generalization of the univariate diffuse-prior regression result.
• Inverse-Wishart posterior for $\Sigma$. The cross-equation variance–covariance matrix $\Sigma$ has an inverted Wishart posterior distribution; the paper records several of its properties (a foundational appearance of the inverse-Wishart as the conjugate/diffuse posterior for a covariance matrix).
• Joint posterior in the general SUR case. For the general model with different regressors per equation, the full joint posterior of the stacked coefficient vector $\beta$ is derived and discussed — the Bayesian counterpart to Zellner's (1962) classical SUR / feasible-GLS estimator.

Concepts Introduced or Extended

• Seemingly Unrelated Regression
• Bayesian Linear Regression
• Covariance Matrix Estimation

Entities Mentioned

• George C. Tiao
• Arnold Zellner

Quotes

"In this paper, we use a Bayesian approach to analyse sets of regression equations with correlated error terms. In the case that the matrix of 'independent variables' is the same for all equations, our model reduces to the traditional multivariate regression model."

"The marginal posterior distribution of the regression coefficient vector for any equation is shown to be of the multivariate-$t$ form … the variances and covariances of the error terms have an 'inverted' Wishart distribution a posteriori."

My Take

This is one of the founding papers of Bayesian multivariate regression, and it fixes the two distributional results that still anchor the whole SUR / Bayesian-VAR literature: multivariate-$t$ marginal posteriors for the coefficients and an inverse-Wishart posterior for the error covariance under a diffuse prior. Those are exactly the conjugate building blocks that later reappear as the Normal–inverse-Wishart posterior of the reduced-form VAR and as the Gibbs blocks of modern SUR samplers. Written the same year as Tiao and Zellner's Biometrika "Bayes's Theorem" paper, it is the multivariate complement to that univariate work: where the diffuse-prior single regression gives a Student-$t$ posterior, the system gives a multivariate-$t$ and an inverse-Wishart. Its limitation is the era's: the same-$X$ special case is fully closed-form, but the general different-regressors SUR posterior is only characterized — the computational tools (Gibbs, direct Monte Carlo) to sample it routinely came decades later.