Harvey-Liu-Zhu (2016) …and the Cross-Section of Expected Returns

Summary

Argues that, after decades of data mining, the conventional $t>2.0$ significance cutoff is far too lax for judging a new asset-pricing factor. Cataloguing the hundreds of factors proposed since 1967 and applying a multiple-testing framework, the authors compute time-varying significance hurdles and conclude that a newly proposed factor should clear a $t$-statistic of about 3.0 (roughly a 0.5% single-test level). Because many tried factors were never published, even this is a lower bound. The provocative headline: most claimed research findings in financial economics are likely false.

Key Claims

• The usual cutoff is obsolete. With hundreds of factors tried, using $t>2.0$ guarantees a flood of false positives; the significance threshold must rise with the number of tests.
• A $t>3.0$ hurdle. Accounting for multiple testing, a credible new factor needs roughly $t>3.0$; the exact hurdle has drifted upward over time as the factor count grew, and the paper provides historical cutoffs from 1967 to the present.
• Three multiple-testing procedures. The factor literature is disciplined with Bonferroni and Holm (family-wise error rate) and Benjamini-Hochberg-Yekutieli (BHY) (false-discovery rate) adjustments, adapted to the finance setting.
• Publication bias worsens it. Since unpublished failed factors are unobserved, the true number of tests exceeds the visible count, so the real hurdle is even higher.
• "Most findings are likely false." Echoing Ioannidis (2005) for medicine, the authors argue a large fraction of published cross-sectional factors would not survive a proper multiple-testing threshold.

Concepts Introduced or Extended

• Factor Zoo

Entities Mentioned

• Campbell R. Harvey
• Yan Liu
• Heqing Zhu

Quotes

"Hundreds of papers and factors attempt to explain the cross-section of expected returns. Given this extensive data mining, it does not make sense to use the usual criteria for establishing significance. ... A new factor needs to clear a much higher hurdle, with a t-statistic greater than 3.0. We argue that most claimed research findings in financial economics are likely false."

My Take

The paper that named and quantified the factor zoo's inference problem, and it did so with a single memorable number — $t>3.0$ — that reshaped how referees and researchers read a claimed factor. Its lasting contribution is less any one procedure than the reframing: factor discovery is a multiple-testing problem, and significance must be judged against the whole search, not a lone regression. It is the cross-sectional counterpart of the authors' own Sharpe-ratio haircut for backtests, and it set the stage for the estimation-based disciplining of the zoo in Feng-Giglio-Xiu (2020). The honest caveat the authors themselves stress — you never see the factors that were tried and quietly abandoned — means $t>3.0$ is a floor, not a ceiling.