Written for a general mathematical audience in the Notices of the American Mathematical Society, this article is the expository companion to the authors' technical work on backtest overfitting. It warns that computational power now lets investment managers search thousands or millions of candidate strategies, so that a profitable-looking rule can be found by chance and then "validated" by a simulation of its historical performance — a pseudo-mathematical argument that collapses out of sample. The paper formalises the Minimum Backtest Length (MinBTL) needed to guard against this and calls on the profession to disclose the number of trials behind any backtest.
"Recent computational advances allow investment managers to methodically search through thousands or even millions of potential options for a profitable investment strategy. In many instances, the resulting strategy involves a pseudo-mathematical argument, which is spuriously validated through a simulation of its historical performance."
This is the paper that took the backtest-overfitting argument out of the finance journals and put it in front of every working mathematician — and its bluntness ("charlatanism") is the point. The technical content is a distilled version of the PBO/CSCV and deflated-Sharpe machinery, but the memorable deliverable is the MinBTL rule of thumb: a one-line way to see that a two-year backtest chosen from a thousand trials is worthless. Its most valuable and least-followed recommendation is procedural — report the number of trials — which, as with the factor zoo, is exactly the information that makes multiple-testing corrections possible and exactly what researchers are least inclined to disclose. As an AMS Notices piece it is short on new theorems and long on advocacy, which is precisely why it travelled.