Global Panel Forecasting — Pooling vs Flexibility

The M5 Claim

Two ways to forecast a panel of related series. Local: fit a separate classical model to each series in isolation, so every model sees only its own history. Global: train one model across all series pooled together, sharing parameters. The M4 and M5 competitions settled this empirically in favour of global machine learning, and the stated mechanism is borrowing strength — the global model learns from far more data than any single series provides. Here the panel is 48 stocks' weekly volatility, 308 observations each, forecast one week ahead.

Where the received wisdom comes from

M4 (2018) ran 100,000 series across six frequencies, drawn from unrelated business and economic domains — and its headline finding was not a win for machine learning. Several pure ML entries finished below a seasonal-naive benchmark. The winner, Slawek Smyl's ES-RNN, was a hybrid: exponential smoothing handling level and seasonality per series, a recurrent network learning what they share. The runner-up, FFORMA, was a feature-weighted combination of classical methods.

M5 (2020) ran Walmart's daily sales — roughly 42,840 series in one coherent hierarchy, with prices, promotions, SNAP days (the US food-assistance benefit whose payment dates move grocery demand) and calendar events as exogenous drivers, and a great deal of intermittent demand — series that are zero most days and occasionally spike, which most forecasting methods handle badly. The verdict flipped hard: all fifty top accuracy entries used machine learning, and the winner was a LightGBM. Global gradient boosting dominated outright.

The reversal in two years is not a change in the methods so much as a change in the regime. M4's series are individually longer and mutually unrelated, so there is little common structure to pool and each series can largely fit itself. M5's are short, numerous, related by a shared calendar and shared shoppers, and carry exogenous features — the wide-and-shallow case where borrowing across the panel is worth a great deal. Which makes "global ML wins" a claim about a regime, and raises the obvious question of which regime a given panel is in.

Reproducing it here

The headline reproduces: a single gradient booster trained across all 48 stocks scores 0.4189 against 0.4417 for a per-stock AR, and beats the local model for 79% of individual stocks. ML wins, as the two previous examples predicted it would.

Which Mechanism Is Doing the Work?

That comparison changes two things at once, though — the data the model sees (one series versus 48) and the model class (linear versus boosted trees) — so it cannot say which one is responsible. The missing cell of the 2×2 is a pooled linear AR: global data, same model class as the local baseline. Adding it is what the whole mechanism claim turns on.

Pooling contributes nothing here. One linear AR fitted to all 48 series scores 0.4430 — very slightly worse than 48 separate ones. Decomposing the total gain: −0.0012 from pooling, +0.0241 from nonlinearity. Diebold–Mariano agrees: local versus global AR is −0.24, indistinguishable, while global AR versus global GBM is +4.02. On this panel the win is flexibility, not borrowing strength.

And when does pooling pay?

Which is not the end of it, because the sweep rescues the M5 claim as a conditional one — and that is a better result than either version. Varying how much history each series gets and tracking the two effects separately, they trade places.

At 20 weeks per series, pooling is worth +0.137 and flexibility −0.006 — the panel is doing all the work and the boosted model is, if anything, a slight liability. At 240 weeks the position is reversed: pooling −0.001, flexibility +0.026. The curves cross somewhere around 50–90 weeks. Borrowing strength across a panel is real and is exactly what M5 found — but it is a remedy for short series, and this panel's 260 weeks each put it on the wrong side of the crossover. The right summary is not "global beats local" but pooling pays when series are short, flexibility pays when they are long — and knowing which regime you are in is the whole decision.

How the Baseline Is Scored

A note on how the classical baseline is scored, because it changes the answer by more than any modelling choice in the notebook. Fitting ETS once and asking for a single 40-step-ahead path — with an additive trend extrapolated across 40 weeks of a mean-reverting log-volatility series — gives 0.757, worse than persistence, and makes the method look broken. Scored the way every other model here is scored, as a genuine one-step forecaster refit on an expanding window, it gives 0.478: competitive, and roughly level with persistence. A comparison is only as honest as its weakest setup, and the protocol is part of the setup.

Where this sits

The same estimation-risk thread as the macro example, seen from the other side: there the problem was too many parameters for one system, here it is too little data per series. And the crossover has a Bayesian reading — pooling is a hierarchical prior, shrinking each series toward a panel mean, which is exactly the mechanism the hierarchical BVAR uses and which pays for the same reason: it helps most when the individual unit is data-poor.

Notebook

Downloads

References

• Makridakis, S., Spiliotis, E. & Assimakopoulos, V. (2022). M5 accuracy competition: results, findings and conclusions. International Journal of Forecasting 38(4), 1346–1364. — the global-model result tested here
• Januschowski, T. et al. (2020). Criteria for classifying forecasting methods. International Journal of Forecasting 36(1), 167–177. — the local/global distinction
• Montero-Manso, P. & Hyndman, R. J. (2021). Principles and algorithms for forecasting groups of time series. International Journal of Forecasting 37(4), 1632–1653. — why global models work, and when
• Diebold, F. X. & Mariano, R. S. (1995). Comparing predictive accuracy. Journal of Business & Economic Statistics 13(3), 253–263. — the test separating the two mechanisms
• Gelman, A. & Hill, J. (2007). Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge. — pooling as a prior, and why it pays for data-poor units