Random-forest variable importance measures (VIMs) are widely used for variable selection — for example, choosing genetic markers to predict a disease. Strobl et al. show that, while VIMs are sensible in many settings, they are not reliable when candidate predictors vary in their scale of measurement or number of categories: suboptimal predictors can be artificially preferred. Simulation studies illustrate the problem and trace it to two mechanisms, and the authors propose a solution — an alternative random-forest implementation of conditional-inference trees grown with subsampling without replacement — that yields unbiased variable selection and reliable VIMs.
cforest in the party package) grown with subsampling without replacement; the resulting VIMs are unbiased and can be used reliably for variable selection even with mixed-type predictors."Random forest variable importance measures … are not reliable in situations where potential predictor variables vary in their scale of measurement or their number of categories … We propose to employ an alternative implementation of random forests, that provides unbiased variable selection in the individual classification trees … using subsampling without replacement."
The paper that turned "random forests give you free variable importance" into "…but read the fine print." Its value is diagnostic precision: the bias is not a vague warning but two identifiable culprits — bootstrap-with-replacement and impurity-based split selection — each with a concrete fix (subsample without replacement; use significance-test-based conditional-inference splits). That makes it the canonical citation for the selection-bias caveat on the wiki's CART page, and a healthy corrective to the enthusiasm for tree-ensemble importance in applied work — including the ML-for-returns literature (Gu-Kelly-Xiu), where importance rankings are routinely reported. The caveat is scope-specific (mixed-type predictors), but exactly where practitioners are least likely to notice it.