Breiman introduces random forests — ensembles of decision trees in which each tree is grown on a bootstrap sample of the data and, crucially, splits each node using a random subset of the features. Letting the trees vote (classification) or average (regression) yields large accuracy gains. He proves that the generalization error converges almost surely as the number of trees grows (so more trees never overfit), and bounds that error in terms of the strength of the individual trees and the correlation between them. Random feature selection lowers inter-tree correlation, giving accuracy competitive with AdaBoost but greater robustness to noise, and out-of-bag data provide internal estimates of error, strength, correlation, and variable importance.
"Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently … The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them."
One of the most consequential applied-statistics papers of its era: random forests are the default "just works" nonlinear predictor, and this paper both defines them and explains why they work through the strength–correlation decomposition — a rare case where a hugely practical method comes with a clean, honest theory of its own error. The out-of-bag machinery is underrated: it delivers cross-validation-quality error estimates and a variable-importance measure for free, which is exactly why forests became the workhorse for feature screening (and why the Strobl et al. caution about biased importance matters so much). On this wiki it is the ensemble capstone on the CART tree, the nonlinear engine behind ML asset pricing, and a natural nuisance learner for double machine learning. Breiman's framing — predictive accuracy over interpretable models — is also the manifesto of his "two cultures" argument.