Noninformative Prior for VAR

Definition

A noninformative prior for a vector autoregression (VAR) is a prior on the coefficient-covariance pair $(\Phi, \Sigma)$ that encodes minimal subjective information, typically chosen to approximate the Bayesian analog of a frequentist analysis. The two most common choices are the constant-Jeffreys prior (also called the diffuse prior) and the constant-reference prior (Yang-Berger 1994).

Key Ideas

Noninformative priors are useful when the researcher lacks prior information or wishes to compare results across studies without hyperparameter choices obscuring the comparison.
Both common priors use a flat (constant) prior on the coefficient matrix $\Phi$ ; they differ only in the prior on the covariance matrix $\Sigma$ .
Improper priors require checking that the posterior is proper before computing Bayes estimates — the sample size conditions differ between the two priors.
The choice of prior matters quantitatively: the constant-Jeffreys prior biases $\Sigma$ estimates upward, while the reference prior tracks the maximum likelihood estimator (MLE) much more closely.
The posterior mean of $\Phi$ is identical (equals the MLE) under both priors; the difference is entirely in how $\Sigma$ is estimated.

How It Works

Model Setup

The $p$ -variable VAR with $L$ lags in compact form is: $Y = X\Phi + \varepsilon, \qquad \varepsilon \sim \mathcal{N}(0, \Sigma \otimes I_T) \tag{VAR}$

where $Y$ is $T \times p$ , $X$ is $T \times (1 + Lp)$ , and $\Phi$ is $(1 + Lp) \times p$ . The MLE is: $\hat{\Phi}_M = (X'X)^{-1}X'Y, \qquad \hat{\Sigma}_M = S(\hat{\Phi}_M)/T, \quad S(\Phi) = (Y - X\Phi)'(Y - X\Phi)$

The Constant-Jeffreys Prior

The standard diffuse (Tiao-Zellner 1964; Geisser 1965) prior is: $\pi_J(\Phi, \Sigma) \propto |\Sigma|^{-(p+1)/2} \tag{J}$

This is the Jeffreys prior for $\Sigma$ alone, combined with a flat prior on $\Phi$ . Under this prior:

Conditional posterior of $\phi = \text{vec}(\Phi)$ given $(\Sigma, Y)$ : $\mathcal{N}(\hat\phi_M,\; \Sigma \otimes (X'X)^{-1})$
Marginal posterior of $\Sigma$ given $Y$ : Inverse Wishart $(S(\hat\Phi_M),\; T - Lp - 1)$
Marginal posterior of $\sigma_{ii}$ : Inverse Gamma with $T - (L+1)p$ degrees of freedom and scale $s_{ii}$

The posterior mean of $\sigma_{ii}$ is $s_{ii}/(T - (L+1)p - 1)$ . Since the unbiased estimator is approximately $s_{ii}/(T - Lp - 1)$ , this over-estimates by: $\text{Bias factor} = \frac{p}{T - (L+1)p - 1} \times 100\% \tag{Bias-J}$

For $L=1, p=5, T=50$ : $\approx 13\%$ . For $L=1, p=5, T=150$ : $\approx 4\%$ . For $L=1, p=10, T=150$ : $\approx 8\%$ . The bias grows with $p$ and shrinks with $T$ .

Direct Monte Carlo (MC) algorithm (Algorithm J): Since the posteriors are conjugate, $(\Phi, \Sigma)$ can be sampled directly — no Markov chain:

Draw $\Omega \sim \text{Wishart}(S(\hat\Phi_M)^{-1},\; T - Lp - 1)$ ; set $\Sigma = \Omega^{-1}$
Draw $\phi \sim \mathcal{N}(\hat\phi_M,\; \Sigma \otimes (X'X)^{-1})$

Posterior propriety: The constant-Jeffreys posterior is proper iff $T \geq (L+1)p + 1$ .

The Constant-Reference Prior (Yang-Berger 1994)

Yang and Berger (1994) derive the reference prior for a covariance matrix $\Sigma$ from a normal population: $\pi_R(\Sigma) \propto \frac{1}{|\Sigma|\displaystyle\prod_{1 \leq i < j \leq p}(d_i - d_j)} \tag{R}$

where $d_1 \geq d_2 \geq \cdots \geq d_p$ are the eigenvalues of $\Sigma$ . The joint prior for $(\Phi, \Sigma)$ uses the same constant prior on $\Phi$ : $\pi_R(\phi, \Sigma) \propto \frac{1}{|\Sigma|\displaystyle\prod_{i < j}(d_i - d_j)}$

The reference prior is motivated by information-theoretic considerations (maximizing the expected Kullback-Leibler divergence between prior and posterior) and is designed to be minimally informative while remaining proper after conditioning on data.

Under the reference prior:

Conditional posterior of $\phi | (\Sigma, Y)$ : same as under Jeffreys — $\mathcal{N}(\hat\phi_M,\; \Sigma \otimes (X'X)^{-1})$
Conditional posterior of $\Sigma | (\phi, Y)$ : non-standard due to the eigenvalue product factor (no closed-form Inverse Wishart)

Posterior propriety: The constant-reference posterior is proper iff $T > Lp + 1$ (slightly weaker than the Jeffreys condition).

Hit-and-Run MCMC Algorithm (Algorithm R)

Because $\pi(\Sigma | \phi, Y)$ has no standard form, Sun-Ni (2005) sample $\Sigma$ via a hit-and-run Metropolis step in the log-eigenvalue space:

Draw $\phi^{(k)} \sim \mathcal{N}(\hat\phi_M, \Sigma^{(k-1)} \otimes (X'X)^{-1})$ ; compute $S_k = (Y - X\Phi^{(k)})'(Y - X\Phi^{(k)})$
Log-transform: let $\Sigma^* = \log(\Sigma)$ via eigen-decomposition $\Sigma = OD O'$ , $\Sigma^* = O D^* O'$ where $d^*_i = \log(d_i)$
Select a random symmetric $p \times p$ direction matrix $V$ (elements $v_{ij} = z_{ij}/\sqrt{\sum_{l \le m}z_{lm}^2}$ , $z_{ij} \sim \mathcal{N}(0,1)$ )
Propose $\Psi = \Sigma^*_{k-1} + \lambda V$ , $\lambda \sim \mathcal{N}(0,1)$ ; decompose $\Psi = Q C^* Q'$
Compute log acceptance ratio $\alpha_k$ from the conditional posterior and accept/reject via Metropolis

Acceptance rate is $\approx 32\%$ in practice; the log-transform ensures $\Sigma$ remains positive definite. This algorithm extends the Gibbs sampler of Algorithm J with one non-conjugate block.

Loss Functions and Bayes Estimators

Sun-Ni (2005) consider two loss functions for $\Sigma$ :

Entropy loss $L_1$ : $\text{tr}(\hat\Sigma^{-1}\Sigma) - \log|\hat\Sigma^{-1}\Sigma| - p$ $\to$ Bayes estimator is the posterior mean $\mathbb{E}[\Sigma|Y]$
Quadratic loss $L_2$ : $\text{tr}(\hat\Sigma\Sigma^{-1} - I)^2$ $\to$ Bayes estimator $\hat\Sigma_2$ solves $\text{vec}(\hat\Sigma_2) = [\mathbb{E}\{(\Sigma^{-1}\otimes\Sigma^{-1})|Y\}]^{-1}\text{vec}(\mathbb{E}[\Sigma^{-1}|Y])$

Under $L_1$ (entropy loss), the constant-Jeffreys Bayes estimator is the posterior mean — exactly where the over-estimation bias appears. Under $L_2$ , the Jeffreys estimator under-estimates, while the reference estimator is again closer to the true value.

Simulation Results

Three Monte Carlo experiments compare the two priors. The PRIAL (Percentage Reduction In Average Loss) of constant-reference vs. constant-Jeffreys under each loss:

$p$	$T$	$\text{PRIAL}_1$ ( $L_1$ )	$\text{PRIAL}_2$ ( $L_2$ )
5	150	65.5%	62.6%
5	50	66.3%	56.9%
10	150	71.8%	66.3%

The reference prior advantage grows with $p$ and persists at small $T$ . In the 10-variable consumption VAR application, constant-Jeffreys variance estimates are $\approx 20\%$ above MLE; reference prior estimates track MLE for large variances.

Shrinkage Prior for $\Phi$ (Ni-Sun 2005)

Ni and Sun (2005) introduce a shrinkage prior on $\phi = \text{vec}(\Phi)$ : $\pi_S(\phi) \propto \|\phi\|^{-(J-2)}, \quad J = p(Lp+1)$ This is proper for J > 2 (always satisfied in VAR contexts) and is implemented via a two-level hierarchy: $\phi \mid \delta \sim \mathcal{N}_J(0,\, \delta I_J), \qquad \pi(\delta) \propto 1$ Integrating over $\delta$ recovers $\pi_S(\phi)$ . The full conditional is inverse-gamma (IG): $\delta \mid \phi, \Sigma, Y \sim \mathrm{IG}(J/2-1,\, \tfrac{1}{2}\phi'\phi)$ . The posterior mean shrinks toward zero: $\mu_S = \delta(\Sigma\otimes(X'X)^{-1}+\delta I)^{-1}\hat\phi_M$ . Simulation results show this prior dominates all constant priors (Jeffreys, RATS, reference) in frequentist risk; prior choice for $\Phi$ matters more than loss function choice.

Linear-Exponential (LINEX) Loss Function

Varian (1975) proposed the asymmetric linear-exponential (LINEX) loss for scalar estimation: $L_{\Phi,2}(\hat\phi_{ij}, \phi_{ij}) = e^{a_{ij}(\hat\phi_{ij}-\phi_{ij})} - a_{ij}(\hat\phi_{ij}-\phi_{ij}) - 1$ The Bayes estimator under this loss is: $\hat\phi_{ij} = -\frac{1}{a_{ij}} \log E\{\exp(-a_{ij}\phi_{ij}) \mid Y\}$ Ni and Sun (2005) set $a_{ij} = -4$ for non-intercept coefficients: the negative sign means underestimation is penalized more heavily, correcting the downward bias of the posterior mean when eigenvalues approach the unit circle.

Student-t Extension (Ni-Sun 2005)

Replace Gaussian errors with a multivariate Student-t via a scale mixture: $\varepsilon_t \mid q_t \sim \mathcal{N}_p(0,\, q_t^{-1}\Sigma), \quad q_t \sim \mathrm{Gamma}(\nu/2,\, \nu/2)$ Conditional on $\{q_t\}$ , the model is normal; each $q_t$ is drawn as an additional Gibbs block. The degrees-of-freedom $\nu$ conditional is log-concave when $T/2 + a - 1 > 0$ , enabling Gilks-Wild (1992) adaptive rejection sampling.

The Normal-Diffuse Prior (Kadiyala-Karlsson 1993)

The Normal-Diffuse prior combines a Minnesota-style informative prior on the regression coefficients with a diffuse (Jeffreys') prior on $\Psi$ :

$\gamma \sim \mathcal{N}(\tilde\gamma,\, \tilde\Sigma), \qquad p(\Psi) \propto |\Psi|^{-(q+1)/2}$

where $\tilde\gamma$ and $\tilde\Sigma$ encode the Minnesota random-walk belief and lag-decay structure (equation-by-equation), but $\Psi$ is left unrestricted by the prior. This resolves the Minnesota prior's key flaw — fixing $\Psi$ as known and diagonal — while retaining the practitioner's ability to specify beliefs about regression coefficients.

The marginal posterior of $\gamma$ is the product of the marginal prior $N(\tilde\gamma, \tilde\Sigma)$ and a matrix- $t$ factor from the likelihood — it can be bimodal when the prior mean and the likelihood mode are far apart, and is not available in closed form. Kadiyala and Karlsson (1993) evaluate it by importance sampling with a 2-0 poly- $t$ density (Drèze 1977) as the importance function.

Empirically, the Normal-Diffuse prior performs best for log gross national product (GNP) in the Canadian small-sample experiment (Exp. 1, 17–26 observations) and best overall in the Canadian large-sample experiment (Exp. 2, 56–75 observations). It is second only to the extended natural conjugate (ENC) prior in the small-sample case, and generally dominates or matches the Minnesota prior.

Why It Matters

The constant-Jeffreys prior is the default in most applied VAR software and virtually all papers that use "diffuse priors." The bias result shows this default is quietly wrong in a quantifiable way.
The reference prior fix is computationally feasible and reduces average loss by two-thirds in typical macro VAR settings.
Propriety of the posterior is non-trivial for improper priors: large-variance normal approximations to flat priors can have badly behaved posteriors even when the true flat prior does not (Hobert and Casella 1996).
This is a distinct issue from informative prior choice (Sims-Zha, Minnesota, steady-state); it applies whenever a researcher claims to use a "noninformative" prior.

Open Questions

A true reference prior for $\Phi$ exploiting the AR coefficient structure is intractable; Ni-Sun (2005) propose a shrinkage prior as an admissible Bayesian alternative, but an information-theoretically motivated prior for $\Phi$ remains open.
A joint reference prior for $(\Phi, \Sigma)$ (rather than a product of separate priors) may further improve performance but is not derivable with current algorithms.
Extension to unit-root or cointegrated VARs: Berger-Yang (1994) found difficulties with noninformative priors in nonstationary autoregressive (AR(1)) models; the VAR case is similarly open.