Multivariate Macro — VAR / BVAR vs Machine Learning

Python · statsmodels · scikit-learn · XGBoost · PyTorch · six FRED series

The Parameter Explosion

The univariate example found classical models leading on a single clean series. This raises the dimension: forecasting six US macro series jointly — industrial production, unemployment, CPI, fed funds, the 10-year yield and payrolls — where each influences the others over time. That is the vector autoregression's domain, and its notorious weakness. With $m=6$ variables and $p=6$ lags the system carries 222 coefficients, estimated from a few hundred noisy monthly observations.

y_t = c + \sum_{\ell=1}^{p} A_\ell\, y_{t-\ell} + \varepsilon_t, \qquad m(mp+1) = 6\times 37 = 222 \text{ coefficients}

The cure is shrinkage, and three flavours line up side by side: a Bayesian VAR with the Minnesota prior (the classic econometric answer, shrinking toward a random walk), Ridge-VAR (its closed-form L2 cousin), and Lasso-VAR (L1, which *selects* a sparse set of lag relationships). Against them, gradient boosting per target and a multivariate LSTM. All are scored on 60 genuine one-step-ahead forecasts using actual lags, so there is no error compounding here.

Against the Right Benchmark

The first thing worth fixing is the yardstick. A persistence baseline is the forecast that repeats the last observed value with no model at all, and it is the floor every table here is measured against. Because these six series are already differenced, the last observed value is last month's change — so persistence asserts that this month's movement repeats next month, which is a real prediction rather than a null one. The natural null is the random walk in levels: predict zero change. Add it and the headline result sharpens considerably. The unrestricted VAR scores 0.6249; predicting nothing at all scores 0.6291. Two hundred and twenty-two estimated coefficients buy a dead heat with doing nothing. That, rather than its position in a table, is the real indictment of over-parameterisation.

avg 1-step OOS RMSE (6 standardized variables, 60 months)	value	DM vs unrestricted VAR
gradient boosting (per target)	0.5819	−2.00 significant
Lasso-VAR (L1, sparse)	0.5846	−2.33 significant
BVAR (Minnesota prior)	0.5887	−2.11 significant
Ridge-VAR (L2)	0.5967	−1.76 — not significant
VAR (OLS, unrestricted)	0.6249	—
random walk in levels (predict zero change)	0.6291	+0.18 — a dead heat
persistence (repeat last month's change — not the same as predicting none)	0.7207
LSTM (multivariate)	0.9131	last, worse than naive

Sixty one-step forecasts are enough to test the ordering rather than read it off. On a Diebold–Mariano test against the unrestricted VAR, the Lasso-VAR (−2.33), the BVAR (−2.11) and the gradient booster (−2.00) all clear the 1.96 threshold. Ridge-VAR does not, at −1.76. So the direction is right — shrinkage helps, trading a little bias for a large cut in variance — but "every shrinkage method beats OLS" claims more than sixty observations can establish.

Where ML Lands

And the claim the section is actually built on holds up cleanly. Gradient boosting tops the table at 0.5819, but it is statistically tied with the shrinkage-linear models: DM −0.23 against the Lasso-VAR and −0.52 against the BVAR, both far inside the threshold. ML is competitive here, where on one clean series it was not — but it does not win. Note also that Lasso-VAR ≈ BVAR (DM −1.35): L1 selection and the Bayesian Minnesota prior reach the same accuracy from opposite traditions, the frequentist and Bayesian faces of one idea.

The multivariate LSTM is the exception in the other direction, and the notebook now says so plainly rather than placing it mid-field: at 0.9131 it finishes last, worse than a random walk. A 150-step full-batch fit is far too little training for a recurrent network, and 796 observations far too little data to justify one.

The per-variable view adds one more caveat. Regressing actual on predicted — the Mincer–Zarnowitz check used in the LSTM example — gives a slope of 0.762 for the BVAR and 0.698 for the booster, both well below the 1.0 an unbiased forecast requires. Both over-react: they move more than their information content justifies, and the booster more so. That is invisible in RMSE and is the kind of bias that matters when a forecast feeds a policy rule or a risk limit.

The Horizon the Exercise Was Avoiding

One more thing the scoreboard cannot show, and it reframes everything above: every number so far is one step ahead. That is the easiest horizon there is — the actual lags are supplied at each step, so nothing compounds and no model propagates its own errors. Repeating the comparison at 1, 3, 6 and 12 months with direct forecasts (a separate fit per horizon, so no recursive bias) shows how narrow the result is.

OOS RMSE by horizon (direct forecasts)	VAR (OLS)	Ridge-VAR	Lasso-VAR	GBM	random walk
1 month	0.625	0.597	0.585	0.582	0.629
3 months	0.651	0.611	0.608	0.621	0.629
6 months	0.679	0.615	0.622	0.726 — now worse than naive	0.629
12 months	1.415 — 2.2× worse than naive	0.674	0.645	0.976	0.629 — wins

At twelve months the random walk beats every model. The unrestricted VAR degrades to 1.415 — more than twice the error of predicting zero — and the gradient booster to 0.976. Only the two shrinkage estimators stay near the naive benchmark, and neither beats it. The gradient booster's advantage is gone by six months, where it is already worse than doing nothing. So shrinkage buys real short-horizon accuracy, and none of these methods buys long-horizon predictability that is not in the data. The one-step contest is worth running, but it is the horizon at which all of these models look their best.

Where this sits

The unifying thread is estimation risk — the danger that comes not from choosing the wrong model but from having to estimate too many quantities from too little data, so that the noise in the estimates swamps whatever the model got right. It is the same problem as in High-Dimensional Portfolios and Shrinking the Factor Zoo — Bayesian priors, L1 selection and L2 all taming the same coefficient explosion, here on VAR coefficients rather than portfolio weights or factor loadings. The Minnesota sampler is reused directly from the Bayesian VAR work, and the sparse Lasso-VAR is the frequentist sibling of SSVS for VARs.

Notebook

Python Six FRED macro series forecast jointly by an unrestricted VAR, Minnesota BVAR, Ridge-VAR, Lasso-VAR, per-target gradient boosting and a multivariate LSTM, with a random-walk-in-levels benchmark and Diebold-Mariano tests on every comparison (Python) Open →

Downloads

INDPRO.csvIndustrial production index (FRED) UNRATE.csvCivilian unemployment rate (FRED) CPIAUCSL.csvConsumer price index, all urban consumers (FRED) FEDFUNDS.csvEffective federal funds rate (FRED) GS10.csv10-year Treasury constant-maturity yield (FRED) PAYEMS.csvTotal nonfarm payroll employment (FRED)

References

Sims, C. A. (1980). Macroeconomics and reality. Econometrica 48(1), 1–48. — the VAR
Doan, T., Litterman, R. & Sims, C. (1984). Forecasting and conditional projection using realistic prior distributions. Econometric Reviews 3(1), 1–100. — the Minnesota prior
Bańbura, M., Giannone, D. & Reichlin, L. (2010). Large Bayesian vector auto regressions. Journal of Applied Econometrics 25(1), 71–92. — shrinkage as the answer to dimension
Diebold, F. X. & Mariano, R. S. (1995). Comparing predictive accuracy. Journal of Business & Economic Statistics 13(3), 253–263. — the test applied to every comparison here
Medeiros, M. C. et al. (2021). Forecasting inflation in a data-rich environment: the benefits of machine learning methods. Journal of Business & Economic Statistics 39(1), 98–119. — where ML does pull ahead in macro