Loh (2011) Classification and Regression Trees

Summary

An overview of classification and regression trees — prediction models built by recursively partitioning the data and fitting a simple model in each partition, displayed as a decision tree. Loh reviews several widely available tree algorithms and compares their capabilities, strengths, and weaknesses on two worked examples, with particular attention to selection bias in split-variable choice, missing-value handling, and the ability to detect interactions.

Key Claims

• What trees are. Classification trees target dependent variables taking finitely many unordered values (loss measured by misclassification cost); regression trees target continuous responses (loss measured by squared error). Both recursively partition the predictor space and fit a simple model per partition.
• Selection bias is the central weakness. CART's greedy, exhaustive search for the best split is biased toward variables that offer more possible splits (many categories or continuous). Loh's own algorithm family (GUIDE, and the earlier QUEST/CRUISE) removes this by selecting the split variable via a statistical significance test before searching for the split point.
• Algorithm comparison. The review contrasts CART, C4.5, and the test-based methods on selection bias, handling of missing values (surrogate splits vs. alternatives), and detection of interactions, illustrated on two data examples.
• Trees as a machine-learning method. Framed as flexible, interpretable prediction models, with their instability and the ensembling remedy noted.

Concepts Introduced or Extended

• Classification and Regression Trees (CART)

Entities Mentioned

• Wei-Yin Loh

Quotes

"Classification and regression trees are machine learning methods for constructing prediction models from data. The models are obtained by recursively partitioning the data space and fitting a simple prediction model within each partition."

My Take

A compact, authoritative map of the tree-algorithm landscape from the person who did the most to fix its original sin. The value of the piece is comparative: it makes clear that "CART" is one point in a design space, and that the greedy exhaustive-search split rule — convenient and famous — is precisely what introduces selection bias, which Loh's test-then-split algorithms (GUIDE/QUEST/CRUISE) were built to avoid. That connects it directly to the Strobl et al. critique of biased forest variable importance: the disease is the same (impurity-based selection favours high-cardinality variables), diagnosed here at the single-tree level and there at the ensemble level. As a survey it is light on new results but heavy on judgement, which is exactly what a reader choosing among tree implementations needs.