← All examples

Machine Learning: Financial ML (López de Prado)

Every other subsection asks how to build a model. This one asks how to stop fooling yourself once you have built one, and it exists because financial data breaks assumptions that most machine-learning tooling takes for granted.

Labels overlap, so ordinary cross-validation hands the model its own test set. Prices are non-stationary, so the standard fix — differencing to returns — throws away the memory the signal lives in. Outcomes are path-dependent, so a fixed-horizon label misrepresents what a trade actually does. And a strategy is rarely the first one tried, so a Sharpe ratio computed on the winner of a search is not the Sharpe ratio of a strategy.

Each tool here answers one of those, and each is checked the way the rest of the collection checks things: in a simulated world where the right answer is known by construction, because a method that only ever runs on real data can never be caught being wrong.

Labels that last, and what they do to cross-validation. Outside finance an observation and its label arrive together. Here a label is the outcome over a forward window — the return over the next ten days, or which price level a trade reaches first — so consecutive labels are built from overlapping stretches of the same history and are strongly correlated. That is what makes ordinary cross-validation unsafe. Shuffling rows scatters each test observation’s near-duplicate neighbours through the training set, and the model is effectively shown its own answers: leakage, information reaching the model at fit time that would not have been available when the forecast was due. Two repairs follow. Purging removes from training every observation whose label window overlaps the test window at all. Embargo additionally drops a block immediately after each test fold, to catch dependence that outlives the label horizon. Scores here are read as AUC — the area under the receiver-operating-characteristic curve, the probability a randomly chosen positive case is ranked above a randomly chosen negative one, so 0.5 is a coin flip and any excess over it, in a world built to contain no signal, is leakage measured directly.

Stationarity, memory, and the quantity in between. Most statistical machinery assumes a stationary series — one whose mean, variance and autocorrelations do not drift over time. Prices are not stationary; they wander, and a model fitted on one price range extrapolates blindly outside it. The standard fix is to difference: model returns instead of levels. That works and it is expensive, because differencing once erases almost all of the series’ memory — the long-range dependence between distant observations, which is where the slow structure lives. Fractional differentiation is the compromise: difference by a fractional amount d, keeping as much memory as stationarity permits. Judging whether the result is stationary is the hard part. The two standard tests point opposite ways — ADF (the augmented Dickey–Fuller test) takes non-stationarity as its null, and KPSS (Kwiatkowski–Phillips–Schmidt–Shin) takes stationarity as its null, so a series is convincingly stationary only when both concur. Neither is built for this case, and the reliable route is to estimate the order of integration directly: a series is said to be I(d) if differencing it d times yields a stationary series, and a fractionally integrated series is stationary exactly when d < 0.5.

Labelling a trade by what happens to it. A fixed-horizon label asks what the return was after ten days, which ignores everything that happened in between — including a stop-loss that would have closed the position on day two. Triple-barrier labelling instead sets three exits, a profit-take above, a stop-loss below and a time limit, and labels the trade by whichever it touches first. The label is therefore path-dependent: it depends on the route the price took, not only on where it ended. Meta-labelling then splits the problem in two — a primary model decides the direction, and a second model decides whether to act on that call at all, learning where the primary is trustworthy. Because the second model only ever declines bets, the metrics move in a particular way: precision (the share of acted-on bets that were right) rises, recall (the share of winning opportunities actually taken) falls, and F1, which combines the two, can stay perfectly flat while the economics change completely.

The last mistake: the strategy you are looking at is the one that survived a search. A Sharpe ratio is return per unit of risk — average return divided by the volatility of those returns, annualised so that daily, weekly and monthly strategies can be compared on one scale. Its weakness is that it says nothing about how many candidates were tried before this one. Test enough configurations on pure noise and the best will post a Sharpe that looks like skill, because the maximum of many random draws is large by construction. Two corrections address this. The deflated Sharpe ratio asks not whether the true Sharpe beats zero but whether it beats the expected maximum from N trials, adjusted for how skewed and heavy-tailed the returns are and how long the record is. The probability of backtest overfitting needs no trial count at all: CSCV (combinatorially symmetric cross-validation) splits the timeline every balanced way, picks the in-sample winner in each split, and records how that winner ranks out of sample. Both come with a subtlety the examples measure rather than assume — the trial count that matters is the number of effectively independent ideas, which on strategies that hold nearly the same positions is far smaller than the number of configurations run.