Kadiyala-Karlsson (1997) Numerical Methods for Estimation and Inference in Bayesian VAR-Models

Summary

Extends the five-prior forecasting comparison of Kadiyala-Karlsson (1993) by providing systematic numerical methods — importance sampling (IS) and Gibbs sampling — for evaluating the Normal-Diffuse and Extended Natural Conjugate (ENC) posteriors that lack closed-form solutions. The paper's decisive finding is that importance sampling becomes infeasible in large vector autoregression (VAR) models: for the 7-variable system, condition (15) fails catastrophically ($T - mk < -159$), driving the relative numerical efficiency of IS toward zero, while Gibbs sampling achieves RNE ≈ 1 at the cost of $O((mk)^3)$ per step. Practical recommendation: Normal-Wishart for standard applications; Normal-Diffuse or ENC when computation is unconstrained.

Key Claims

Methods Compared

Four simulation approaches are evaluated across 3- and 7-variable Canadian macro VARs:

• 2-0 poly-$t$ importance sampler: KK 1993 IS function; drawn via Bauwens-Richard (1982) decomposition algorithm; product of multivariate-$t$ kernels.
• STFC importance function: product of conditional multivariate-$t$'s evaluated at the posterior mode; $O((mk)^{3/2})$ per draw; more accurate IS function than 2-0 poly-$t$ for moderate dimensions.
• Normal-Diffuse 2-block Gibbs: two closed-form full conditionals; $O((mk)^3)$ per step.
• ENC equation-by-equation Gibbs: Drèze-Richard (1983) Lemma 6.4; $m$ multivariate-$t$ draws per cycle; $O(m(mk)^2)$ per cycle.

Normal-Diffuse 2-Block Gibbs Sampler

The prior $\gamma \sim N(\tilde\gamma, \tilde\Sigma)$, $p(\Psi) \propto |\Psi|^{-(q+1)/2}$ yields two conjugate conditionals:

Block 1 — $\gamma \mid \Psi, Y$: $$\gamma|\Psi, Y \sim N\!\left(\bar\gamma,\; (\tilde\Sigma^{-1} + \Psi^{-1} \otimes Z'Z)^{-1}\right)$$ $$\bar\gamma = (\tilde\Sigma^{-1} + \Psi^{-1} \otimes Z'Z)^{-1}\!\left(\tilde\Sigma^{-1}\tilde\gamma + (\Psi^{-1} \otimes Z')y\right)$$

Block 2 — $\Psi^{-1} \mid \gamma, Y$: $$\Psi^{-1}|\gamma, Y \sim W\!\left(S(\Phi)^{-1},\; T\right), \quad S(\Phi) = (Y - Z\Phi)'(Y - Z\Phi)$$

Dominant cost: Cholesky of the $mk \times mk$ precision matrix in Block 1, giving $O((mk)^3)$ per step.

ENC Equation-by-Equation Gibbs Sampler

Under the Drèze-Morales (1976) ENC reparametrization, the full conditional of each equation's coefficients $\gamma_i$ given all others $\gamma_{\neq i}$ is a multivariate $t$ by Drèze-Richard (1983) Lemma 6.4: $$\gamma_i | \gamma_{\neq i}, Y \sim t_k\!\left(\mathbf{d}_i,\; \frac{q_{ii}}{T+\alpha-k}\,\mathbf{P}_{ii}^{-1},\; T+\alpha-k\right)$$ One cycle over all $m$ equations costs $O(m(mk)^2)$.

Relative Numerical Efficiency (RNE)

$$\text{RNE}(M) = \frac{\text{Var}_p[g(\theta)]}{\sigma^2_{g,M}}$$ Measures how many direct draws method $M$ is equivalent to in terms of estimator variance. Gibbs achieves RNE ≈ 1 in all experiments. IS achieves moderate RNE in the 3-variable model but collapses to RNE ≈ 0 in the 7-variable model when condition (15) — $T - mk \geq 0$ — fails ($T - mk = -159$ in the reported experiment).

Antithetic Variates (GAIN)

For importance samplers, drawing antithetic pairs $z^{*(i)} = -z^{(i)}$ (sign-flipped Normal variate through the mode) reduces variance: $$\text{GAIN} \equiv \frac{\text{Var}(\bar g_{2n})}{\text{Var}(\bar g_{a,n})} = \frac{1}{1+\rho}, \quad \rho = \text{Corr}\!\left(g(\theta^{(i)}),\, g(\theta^{*(i)})\right)$$ For near-symmetric posteriors $\rho \approx -1$ and GAIN $\gg 1$. See also Geweke (1988).

CPU Scaling

Method	Cost per draw/step	7-var / 3-var overhead
Normal-Wishart / Diffuse	$O(mk)$	~4×
STFC importance function	$O((mk)^{3/2})$	~8×
ENC equation Gibbs	$O(m(mk)^2)$	~30×
Normal-Diffuse 2-block Gibbs	$O((mk)^3)$	~140×

Full comparative experiment required >200 CPU hours (1997 hardware) per prior.

Recommendation

• Normal-Wishart: first choice for applied work; $O(mk)$ cost; closed-form posterior moments for impulse response functions (IRFs) and variance decompositions.
• Normal-Diffuse or ENC: appropriate when computation is unconstrained; Gibbs sampling feasible for up to 7-variable systems; IS infeasible beyond ~5 variables.
• Importance sampling: viable only for small systems where condition (15) holds; antithetic variates improve efficiency.

Concepts Introduced or Extended

• Gibbs Sampler — Normal-Diffuse 2-block Gibbs and ENC equation-by-equation Gibbs for Bayesian VAR (BVAR); RNE as computational efficiency metric; GAIN from antithetic variates
• Vector Autoregression — computational feasibility of inference under different priors at scale
• Minnesota Prior — identified as computationally preferred ($O(mk)$) despite forecasting inferiority to richer priors
• Noninformative Prior for VAR — computational feasibility of ENC and Normal-Diffuse posteriors in large systems

Entities Mentioned

• K. Rao Kadiyala
• Sune Karlsson

Quotes

"The results indicate that Gibbs sampling constitutes a feasible approach to Bayesian inference even in relatively large VAR models, while importance sampling fails for our largest model."

My Take

The paper's headline result — IS infeasibility for 7-variable systems — is clearly documented and the condition (15) diagnostic is practically useful. The 2-block Normal-Diffuse Gibbs is elegant (both conditionals are standard distributions) but the $O((mk)^3)$ cost limits it to small systems. The ENC equation-by-equation Gibbs is computationally smarter ($O(m(mk)^2)$ via the Drèze-Richard lemma) but harder to implement. In retrospect, the paper correctly identified that the Normal-Wishart's closed-form posterior is its decisive advantage: by the 2000s virtually all central bank BVAR work used Normal-Wishart or conjugate priors precisely because the computational cost of richer priors was prohibitive. The CPU comparison table is a vivid reminder that the BVAR literature's practical choices were shaped as much by computational constraints as by prior theory.