Zellner-Ando (2010) A Direct Monte Carlo Approach for Bayesian Analysis of the Seemingly Unrelated Regression Model

Summary

Develops a Direct Monte Carlo (DMC) approach for Bayesian analysis of the Seemingly Unrelated Regression (SUR) model, published in Journal of Econometrics 159 (2010): 33–45. The key insight is that a triangular reparameterization (Zellner et al. 1988) renders all posterior conditionals exactly normal–inverse-gamma, enabling closed-form sequential sampling with 100% acceptance rate and zero autocorrelation — bypassing Gibbs sampling entirely. Monte Carlo experiments show Gibbs fails to converge 92/100 trials when predictors are numerous; DMC always succeeds and runs 3$\times$ faster.

Key Claims

• The SUR model consists of $m$ regression equations $y_j = X_j \beta_j + u_j$ with correlated errors $E[u_i u_j'] = \omega_{ij}I$. Standard Bayesian analysis via Gibbs requires iterating between $\beta|\Omega$ and $\Omega|\beta$ blocks, which depend on each other, generating autocorrelated draws.
• The triangular transformation (Zellner et al. 1988) rewrites the model as $y_j = Z_j b_j + e_j$ where $Z_j$ augments $X_j$ with residuals from earlier equations. Under Jeffreys' prior $\pi_3 \propto \prod_j (\sigma_j^2)^{-1}$, the conditionals decouple: $b_j | b_{j-1},\ldots,b_1, \sigma_j^2, D \sim \mathcal{N}(\hat{b}_j, \sigma_j^2 (Z_j'Z_j)^{-1})$ and $\sigma_j^2 | b_{j-1},\ldots,b_1, D \sim \text{IG}(\hat\lambda_j/2, \hat\nu_j/2)$ (where $\text{IG}$ denotes the inverse-gamma distribution) — all closed form and independent across equations.
• DMC1 samples directly in the transformed space $\{b, \Sigma\}$ (Jeffreys' prior $\pi_3$); DMC2 adds a transform-back step using the recursive relation $\omega_j^2 = \sum_k \rho_{jk}^2 \omega_k^2 + \sigma_j^2$ to recover $\{\beta, \Omega\}$ under the invariant prior $\pi_1 \propto |\Omega|^{-(m+1)/2}$. Crucially, $\pi_1$ and $\pi_3$ are not the same prior — the Jacobian $|J_m| = \prod_j (\sigma_j^2)^{m-j}$ is non-trivial.
• DMC advantages over Gibbs: no burn-in period, no convergence checking, no proposal density, independent draws (Inefficiency Factor (INEF) = 1 theoretically). Empirically: INEF for $\omega_{12}$ = 31.60 under Gibbs vs. 1 under DMC; Gibbs failed Geweke (1992) convergence test 92/100 trials when $p_j = 25$, $n = 50$; computational time DMC 50.84s vs. Gibbs 168.73s.
• Model selection: Bayesian Predictive Information Criterion (BPIC; Ando 2007) and Deviance Information Criterion (DIC; Spiegelhalter et al. 2002) are both computable in closed form under the DMC posterior samples, enabling variable selection among SUR equation specifications without Markov Chain Monte Carlo (MCMC) overhead.
• Application: Japanese firm sales (automobile, agriculture, service sectors) quarterly 1993–2006; one-quarter-ahead recursive forecasts from 1999:Q2 to 2004:Q4; all 95% predictive intervals contain the realized values.

Concepts Introduced or Extended

• Seemingly Unrelated Regression
• Gibbs Sampler

Entities Mentioned

• Arnold Zellner
• Tomohiro Ando

Quotes

"It is concluded from these comparisons that the DMC approach is worthwhile and applicable to many SUR and other problems."

My Take

An elegant contribution showing that a well-known algebraic trick (triangular decomposition) eliminates the Gibbs bottleneck entirely for a widely used multiequation model. The high-dimensional convergence failure result (92/100 Gibbs failures at $p_j=25$, $n=50$) is striking and practically important. The prior non-equivalence ($\pi_1 \neq \pi_3$) is clearly acknowledged. Limitations: the approach is specific to the SUR linear model; nonlinear or heavy-tailed SUR variants require separate treatment (the authors note Student-$t$ SUR as ongoing work). The empirical application is illustrative rather than a demanding test — a macro forecasting exercise would better stress-test the predictive claims.