Meta-Learners

Four Ways to Wire the Same Regression

Causal forests estimate $\tau(x)$ with a purpose-built algorithm. Meta-learners take the opposite route: use whatever regression method you already trust, and arrange it so that a difference of predictions estimates the effect. The interest is not in any one of them but in the fact that the arrangement matters enormously — the same base learner, wired four ways, produces four different answers.

The S-learner fits one model on all the data with treatment as a feature, and takes $\hat\mu(x,1)-\hat\mu(x,0)$. Simple, and it can compress the effect: if the learner regularizes the treatment indicator toward irrelevance, the estimated effect shrinks toward zero.

The T-learner fits separate models to treated and control and differences them. That avoids the compression, and pays for it whenever one arm is small — the model for the small arm is fit on little data and its noise passes straight into the difference. The X-learner (Künzel et al.) is built exactly for that case: impute each unit's missing counterfactual using the other arm's model, then regress the imputed effects, weighting by the propensity so the better-estimated arm dominates. The R-learner is the orthogonalized one, regressing outcome residuals on treatment residuals — principled, and the most propensity-sensitive of the four.

On a Simulation, the Theory Holds

On a confounded simulation with a known $\tau(x) = x_1 + 0.5x_2$ and a deliberately unbalanced 30% treated arm, they separate as the theory predicts:

True τ(x) = x₁ + 0.5x₂, ATE 0.76, 30% treated, confounded.

The X-learner wins, which is its reason for existing: the small treated arm is exactly the condition it was designed for, and the T-learner — which has to fit that arm alone — is the one it beats. The R-learner comes last despite being the most theoretically principled, because its pseudo-outcome divides by a treatment residual and small residuals amplify noise. A cross-check against econml's implementation agrees with the from-scratch X-learner to a correlation of 1.00.

On the Benchmark, It Inverts

Then the same four on IHDP — the field's standard benchmark, with 747 units, 25 real covariates from an infant-health study, a simulated outcome so the truth is known, and 19% treated with genuinely poor overlap. This is where the exercise earns its keep.

747 units, 25 real covariates, 19% treated, treated propensity around 0.17.

The ranking inverts. The X-learner that won the simulation drops to third; the S-learner that placed second from bottom wins outright; the R-learner does not merely lose but fails, at nearly four times the next-worst error. Poor overlap is the mechanism — with treated propensities around 0.17, the propensity-weighted learners are dividing by small numbers, and the simplest learner, which never touches a propensity, is left standing.

The R companion makes this sharper rather than confirming it. Running the same four learners in grf, the ranking flips between simulation and IHDP again — but the details differ from Python's: there the T-learner leads on IHDP and the S-learner leads in Python; on the simulation R has T ahead of S and Python has S ahead of T. So the ordering is not stable across datasets or across implementations of the same four estimators.

Which is the finding worth carrying. There is no universal winner, and the ranking is not a property of the learners — it is a property of the learners crossed with the overlap in your data. The literature's habit of naming a best meta-learner from one benchmark is exactly the inference this page shows failing. What transfers is not the ranking but the diagnosis: check overlap first, and let it tell you which family is even admissible.

Where this sits

The four learners estimate the same object as causal forests, and the R-learner's residual-on-residual construction is double machine learning applied pointwise. The overlap problem that decides the ranking is the same one that modern balancing pays for in effective sample size, and the propensity score all but the S-learner depend on is the object a coherent Bayesian cannot use.

Notebooks

Downloads

References

• Kunzel, S. R., Sekhon, J. S., Bickel, P. J. & Yu, B. (2019). Metalearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the National Academy of Sciences 116(10), 4156–4165. — S-, T- and X-learners
• Nie, X. & Wager, S. (2020). Quasi-oracle estimation of heterogeneous treatment effects. Biometrika 108(2), 299–319. — the R-learner
• Hill, J. L. (2011). Bayesian Nonparametric Modeling for Causal Inference. Journal of Computational and Graphical Statistics 20(1), 217–240. — the Bayesian alternative