Definition
Machine learning in asset pricing applies flexible, regularized predictive methods — penalized linear models, dimension reduction, regression trees, and neural networks — to the canonical problem of measuring asset risk premiums: forecasting the cross-section of expected stock returns from a large set of predictors. Gu, Kelly & Xiu (2020) provide the benchmark comparative study, finding large economic gains and tracing them to nonlinear predictor interactions (Gu-Kelly-Xiu 2020).
Key Ideas
- The prediction problem. Model the risk premium as Et[ri,t+1]=g⋆(zi,t) for a flexible function g⋆ of stock characteristics and macro variables; the "factor zoo" of ~90+ characteristics (interacted with macro predictors and industry dummies) yields a high-dimensional z.
- A method menu. The comparison spans ordinary least squares, penalized linear (lasso/elastic net), dimension reduction (principal-components and partial-least-squares regression), generalized linear (splines), regression trees (random forests, gradient-boosted trees), and neural networks — all tuned by out-of-sample validation to control overfitting in the large-p setting.
- Trees and neural nets win. The best-performing methods are tree ensembles and neural networks; their edge is traced to allowing nonlinear predictor interactions that linear and dimension-reduction methods miss.
- Large economic gains. ML forecasts deliver substantial out-of-sample value — in some cases doubling the Sharpe ratio of leading regression-based strategies.
- Agreement on signals. Despite different functional forms, all methods agree on the same dominant predictors — variations on momentum, liquidity, and volatility. Variable importance is measured by the drop in predictive R2 from zeroing a predictor and by sum-of-squared-derivatives (mean-decrease-in-impurity for trees).
Why It Matters
- Beyond linear factor models. It moves empirical asset pricing past the linear factor-model paradigm, showing that nonlinearities and interactions among characteristics carry real, tradeable predictive content.
- Complement to the factor zoo. Where Feng-Giglio-Xiu discipline which factors are real via valid inference, Gu-Kelly-Xiu ask how flexibly to combine many predictors — two sides of taming high-dimensional return prediction.
- Template for ML in finance. The out-of-sample, regularization-first methodology (validation tuning, honest Roos2, economic-value metrics) became the standard recipe for machine-learning studies of returns.
Open Questions
- Economic interpretation vs. black boxes. Trees/neural nets predict well but are hard to interpret; reconciling predictive power with economic structure (no-arbitrage, risk compensation) is open.
- Overfitting and non-stationarity. Return predictability is weak and time-varying; guarding against data-snooping and regime change is central.
- From prediction to a stochastic discount factor. Extending characteristic-based prediction to a coherent SDF/risk model (e.g. autoencoder/IPCA latent factors) is an active frontier.
Related