The Horseshoe BVAR

Python · R · Part 12 of the VAR arc — global-local adaptive shrinkage

Overview

Part 12 of the VAR arc — adaptive shrinkage. Every Bayesian VAR in this series shrank with the Minnesota prior: a fixed structure that treats all own-first-lag coefficients alike, all cross-lags alike, and tightens mechanically with lag length. It works, but it cannot learn which coefficients matter — it imposes the same shrinkage whether a coefficient is a genuine dynamic or pure noise. Global-local shrinkage priors — the horseshoe (Carvalho–Polson–Scott 2010) foremost — do learn: a single global parameter pulls everything toward zero, while heavy-tailed local parameters let a few genuinely important coefficients escape. The result is automatic, adaptive sparsity. This example builds the horseshoe VAR from scratch, fits it to a 10-variable quarterly macro system (a VAR(2), 200 coefficients), and asks whether adaptive shrinkage beats the Minnesota rule — and what it reveals that Minnesota hides.

The horseshoe prior

Give each coefficient a normal prior whose variance is the product of a local and a global scale — both half-Cauchy. The global $\tau$ shrinks the whole equation; the local $\lambda_j$ , with heavy tails, occasionally take large values that let individual coefficients slip free of the global pull. Writing the implied shrinkage weight $\kappa_j = 1/(1+\lambda_j^2\tau^2)$ ( $\kappa=0$ escapes, $\kappa=1$ crushed to zero), the horseshoe induces a $\text{Beta}(\tfrac12,\tfrac12)$ prior on $\kappa_j$ — U-shaped, piling mass at 0 and 1. Hence the name: coefficients are either kept nearly intact or crushed, with little in between — exactly what a fixed Minnesota variance cannot produce. Sampling from scratch: the half-Cauchy scales are written as inverse-gamma mixtures (Makalic–Schmidt 2016), so every full conditional is inverse-gamma or normal — a fast conjugate Gibbs sampler, run equation by equation.

\beta_j \sim \mathcal N\!\big(0,\ \lambda_j^2\,\tau^2\,\sigma^2\big), \qquad \lambda_j \sim C^+(0,1), \qquad \tau \sim C^+(0,1)

Results

Radical sparsity. Of the 200 coefficients in the 10-variable VAR(2), the horseshoe kills ~96% ( $\kappa>0.9$ ) and lets only a handful escape — and those it keeps are exactly the ones economics expects to persist: the own first lags of the interest rates and unemployment ( $\kappa\approx0.08$ for unemployment, $0.23$ for the 10-year yield). The near-unpredictable growth-rate equations are shrunk almost entirely away. Without being told anything about the variables, the prior performs automatic variable selection, separating persistent dynamics from noise — something Minnesota's uniform, rule-based shrinkage never surfaces. But the sparsity is not free. Measured honestly — an expanding-window, refit-every-quarter, 1-step density test over 2011–2019 — the horseshoe ( $-14.16$ ) forecasts significantly worse than the Minnesota prior ( $-13.85$ ; paired- $t$ $p\approx0.015$ ), and even a near-flat prior ( $-13.96$ ) edges it. The same aggressive shrinkage that yields the sparsity over-shrinks coefficients carrying genuine short-horizon signal — a known tension, since the horseshoe optimises for selection, not prediction. What it buys instead is interpretability: a radically smaller, readable model that names which dynamics are real, for a small but real forecast premium — which is why global-local priors are a modern default for large, high-dimensional VARs, where the regularisation matters more and the gap typically narrows (Huber–Feldkircher 2019; Cross–Hou–Poon 2020). The R notebook reproduces the same equation-level selection from a base-R implementation of the sampler.

Notebooks

Python 12 · The horseshoe BVAR — from-scratch Makalic–Schmidt Gibbs on a 10-variable VAR: the adaptive shrinkage map, the U-shaped horseshoe, and a forecast comparison against Minnesota and a near-flat prior (Python) Open → R 12 · R cross-check — the same global-local Gibbs from scratch in base R, on the unemployment equation Open →

Downloads

horseshoe.py From-scratch horseshoe VAR — the Makalic–Schmidt auxiliary-variable Gibbs sampler, equation by equation (NumPy / SciPy) bvar.py The Minnesota-prior BVAR module reused from the VAR arc, as the shrinkage benchmark largesv_data.csv 10 stationary FRED-QD macro series, quarterly 1960–2019 (growth rates, inflation, unemployment, the funds rate and the 10-year yield)

References

Carvalho, C. M., Polson, N. G. & Scott, J. G. (2010). The horseshoe estimator for sparse signals. Biometrika 97(2), 465–480. — the global-local prior implemented here
Makalic, E. & Schmidt, D. F. (2016). A simple sampler for the horseshoe estimator. IEEE Signal Processing Letters 23(1), 179–182. — the inverse-gamma auxiliary-variable representation that makes the Gibbs sampler fully conjugate
Huber, F. & Feldkircher, M. (2019). Adaptive shrinkage in Bayesian vector autoregressive models. Journal of Business & Economic Statistics 37(1), 27–39. — global-local shrinkage for the VAR
Cross, J. L., Hou, C. & Poon, A. (2020). Macroeconomic forecasting with large Bayesian VARs: global-local priors. International Journal of Forecasting 36(3), 899–915. — large-VAR forecasting evidence for these priors