Univariate Forecasting — Classical vs ML vs Neural

The Classical Models' Home Ground

The question this section exists to answer, on the easiest possible version of it: do machine-learning forecasters beat classical econometric ones? The data is US Advance Retail Sales (FRED RSXFSN, monthly 1992–2026, deliberately not seasonally adjusted), and the task is a 24-month-ahead forecast of the log series. It has a steady upward trend and a hard annual spike every December, which is exactly the structure the classical methods were built for. SARIMA (seasonal autoregressive integrated moving average) is the standard tool for a series with a trend and a repeating yearly cycle, and exponential smoothing is its close relative. So this is the classical models' home ground, and the bar is meant to be high.

Classical: SARIMA chosen by auto-ARIMA, and ETS/Holt–Winters. Neither needs feature engineering; hand them the series and the seasonal period. ML: a Lasso autoregression and a gradient booster, fed hand-built lags, Fourier seasonal terms and a trend index, rolled forward recursively. Neural: an LSTM (long short-term memory, a recurrent network with gates that let it carry information across long gaps) on the raw windowed series. The asymmetry is the point — the seasonality SARIMA infers on its own has to be constructed by hand for everything else.

At the obvious forecast origin the classical models win: SARIMA 0.017 and ETS 0.020 against the Lasso's 0.032, the gradient booster's 0.060 and the LSTM's 0.080. That is the M-competition result in miniature, and it is where this comparison usually stops. Three things are worth doing before accepting it.

A Broken Challenger, Found in the Coefficients

First: one of the challengers depends entirely on a preprocessing choice, and the error metric never says so. Fitted to the raw design the Lasso scores 0.085 — last, behind a seasonal-naive baseline. Its coefficients say why, where its RMSE (root mean squared error, the accuracy metric used throughout) does not. L1 — the Lasso's penalty, which shrinks coefficients and sets the least useful ones to exactly zero, so that it selects features as well as fitting them — zeroes nine of ten. Every lag, every Fourier seasonal term. The only survivor is the trend index, so the "Lasso autoregression" is not an autoregression at all but a bare straight line — which is exactly what its forecast looks like.

The cause is scale, not L1. The trend index has standard deviation 108.8; the lags sit near 0.36 and the Fourier terms are bounded in [−1,1]. LassoCV does not standardise its design — rescale every feature to a comparable size before penalising them — so a single penalty applied across a 300× spread in scale annihilates the small ones. The penalty is the same for every coefficient, but a feature measured in large units needs only a tiny coefficient to matter, and a tiny coefficient is what the penalty removes first. Standardizing first restores all four seasonal terms and takes the out-of-sample RMSE from 0.085 to 0.032 — from worst in the field to third, ahead of the gradient booster. A comparison is only as honest as the weakest setup in it, and this one is invisible in the accuracy numbers.

A Feature That Does Nothing

Before drawing the conclusion, one mechanism is worth isolating, because it is specific and fixable-sounding. The gradient booster is handed a raw trend index as a feature. In training it runs 13 to 389; across the forecast window it runs 390 to 413 — entirely outside anything the tree ever split on. Holding every other feature fixed and moving only that index, the prediction changes by +0.00000 across the whole forecast window, against +0.149 over the last sixty in-sample steps. A tree emits a constant beyond its final split, so the trend feature is dead the moment forecasting begins and the booster is left extrapolating a trending series with no trend term at all.

The obvious remedy is to model differences instead of levels — and it is worth reporting that it does not rescue the comparison: a differenced booster with no trend index scores about the same. The tree's difficulty is not only extrapolation. A 377-point training set is thin for a flexible learner asked to discover seasonality that SARIMA is handed by construction.

And a network that flatlines

Third: the LSTM does not merely trail, it flatlines — and the reason is structural rather than a tuning failure. Across 24 months its forecast moves 0.005 while the actual series moves 0.276 — in logs, where a move of 0.01 is about 1%, so the series climbs roughly 28% and the forecast is flat to within half a percent. A recursive forecast is a dynamical system: the network maps a window to the next value, that value is appended, and the map runs again. This one has an attracting fixed point. Feed it a constant window at 13.19 and it returns 13.23; at 13.39 it returns 13.34 — pulling inward from both sides toward roughly where the series ends. You can watch the input window's standard deviation collapse from 0.055 to 0.002 as it fills with the model's own output. Nothing sustains an oscillation because the network never learned the seasonal cycle, and training ten times longer lowers in-sample error while leaving the forecast just as flat and out-of-sample RMSE slightly worse.

One Origin Is Not an Evaluation

Now the part that changes the conclusion. Everything above rests on one 24-month window ending at the last observation, and a forecast origin is a draw. Refitting every model at five rolling origins, three different models take the top spot — and the window the headline table uses is the one where the classical models look strongest.

Averaged over origins the direction survives: ETS 0.054 and SARIMA 0.056 lead the standardized Lasso's 0.064, the booster's 0.082 and the seasonal-naive baseline's 0.105. Classical really does win on this kind of series. But "wins big" describes the luckiest origin rather than the evidence — the Lasso wins outright at two of the five origins, and the seasonal-naive baseline that looked competitive at the headline origin is the worst model on average, a property of the window rather than of the method. This is the honest form of the M-competition claim, and the reason those competitions score dozens of origins across thousands of series rather than reporting one holdout.

What this does and does not establish

None of which says ML loses on time series — it says the advantage is conditional. The later examples move to the regimes where the conditions change: many interacting series where shrinkage manages the parameter explosion, a panel of many related series forecast by one model, and financial returns where the target is barely predictable at all.

Notebook

Downloads

References

• Makridakis, S., Spiliotis, E. & Assimakopoulos, V. (2018). Statistical and machine learning forecasting methods: concerns and ways forward. PLoS ONE 13(3). — the finding reproduced and qualified here
• Box, G. E. P. & Jenkins, G. M. (1970). Time Series Analysis: Forecasting and Control. Holden-Day. — SARIMA
• Hyndman, R. J. & Athanasopoulos, G. (2021). Forecasting: Principles and Practice, 3rd ed. OTexts. — ETS, and rolling-origin evaluation done properly
• Tashman, L. J. (2000). Out-of-sample tests of forecasting accuracy: an analysis and review. International Journal of Forecasting 16(4), 437–450. — why one origin is not an evaluation
• Januschowski, T. et al. (2020). Criteria for classifying forecasting methods. International Journal of Forecasting 36(1), 167–177. — local versus global, the axis the later examples move along