Chib-Greenberg (1995b) Hierarchical Analysis of SUR Models

Summary

Chib and Greenberg (1995b) develop a unified Markov chain Monte Carlo (MCMC) framework for Bayesian inference in hierarchical extensions of Zellner's (1962) seemingly unrelated regression (SUR) model. The paper has three main contributions: (1) Gibbs sampling for a three-level hierarchical SUR with conjugate full conditionals; (2) extension to SUR with first-order vector-autoregressive (VAR(1)) or vector-moving-average (VMA(1)) correlated errors, where the moving-average case requires Metropolis-within-Gibbs with a Taylor-approximation candidate density; and (3) a time-varying-parameter (TVP) SUR model estimated via a forward Kalman filter followed by backward simulation of the joint state path — an early derivation of what is now called the Carter-Kohn forward-filter backward-sampling (FFBS) smoother. All methods are illustrated on Organisation for Economic Co-operation and Development (OECD) gross national product (GNP) growth data for five countries (1960–1987).

Key Claims

• The three-level hierarchical prior ($\beta \to \beta_0 \to \mu$) with Wishart precision priors yields all full conditionals in standard form (Normal/Wishart), guaranteeing convergence by the Roberts-Smith (1993) sufficient conditions.
• For VAR(1) errors, the AR matrix $\Phi$ has a matrix-normal full conditional truncated to the stationarity region; accept-reject sampling within Gibbs handles the truncation efficiently.
• For VMA(1) errors, the MA matrix $\Theta$ lacks a standard conditional; a first-order Taylor expansion of the innovations around the nonlinear least squares (NLS) estimate yields a normal candidate generating density (eq. 12) with ${\sim}50\%$ acceptance rate in simulations.
• The key insight for TVP-SUR: simulating all time-varying states $\{\theta_t\}$ jointly (eq. 16) via the factorisation $p(\theta_n|\{\beta_t\}) \cdot p(\theta_{n-1}|\{\beta_t\},\theta_n) \cdots$ gives one Gibbs block instead of $n+1$ blocks; convergence dramatically faster than single-move sampling.
• The FFBS backward-simulation recursion (eq. 17–19): run the Kalman filter forward saving $\{\hat\theta_{t|t}, R_{t|t}, M_t\}$; draw $\theta_n \sim \mathcal{N}(\hat\theta_{n|n}, R_{n|n})$; then draw $\theta_t$ backward from $\mathcal{N}(\hat\theta_t, R_t)$ where $\hat\theta_t = \hat\theta_{t|t} + M_t(\theta_{t+1} - \hat\theta_{t+1|t})$ and $M_t = R_{t|t} \cdot R_{t+1|t}^{-1}$.
• Partial Bayes factors (Dawid 1984): train on the first $n_0$ observations to form a proper posterior; the remaining partial marginal density is estimated as $N^{-1}\sum f(Y_{n_1}|\psi^{(i)}, M_k)$ (eq. 7) — avoids the expensive full Gibbs-based Bayes factor computation.
• OECD GNP application: strong evidence for the pooled (shrinkage) model over the unpooled model (log partial Bayes factor $\approx 40$); weak evidence for time-varying parameters ($\phi$ posterior concentrated near zero).

Concepts Introduced or Extended

• Seemingly Unrelated Regression
• Bayesian Hierarchical Model
• Gibbs Sampler
• Metropolis-Hastings Algorithm
• State-Space Representation
• Time-Varying Parameter VAR
• Marginal Data Density

Entities Mentioned

• Siddhartha Chib
• Edward Greenberg
• Arnold Zellner

Quotes

"Despite what may be expected, sampling the joint distribution in (16) is not difficult. To begin with, we write the joint density of the $\{\theta_t\}$ in reverse time order as $p(\theta_n|\{\beta_t\}, \psi) \times p(\theta_{n-1}|\{\beta_t\}, \theta_n, \psi) \times \cdots \times p(\theta_0|\{\beta_t\}, \theta_1,\ldots,\theta_n, \psi)$."

"As far as we know, there are no other examples of so complex a TVP model being estimated by full Bayesian methods."

My Take

This paper is best read alongside Carter-Kohn (1994); together they establish the FFBS as the workhorse for Bayesian state-space models. The SUR setting here is somewhat incidental — the TVP contribution is the lasting one. The Metropolis-within-Gibbs treatment of VMA(1) errors is an early application of what Chib-Greenberg (1995a) had just formalised theoretically. The partial Bayes factor proposal is elegant but rarely used in practice; the Chib (1995) identity and its Chib-Jeliazkov (CJ) extension proved more popular for model comparison. The OECD application is modest (n=28 years), but the 40-parameter TVP model demonstrates feasibility of MCMC for high-dimensional latent state problems at a time when this was non-trivial.