Policy Learning

An Estimate Is Not a Decision

Causal forests and meta-learners both end with an estimated $\hat\tau(x)$ and stop there. That is not a decision. A programme has to choose whom to enrol, under a budget, and the step from a surface of estimated effects to a rule someone can administer is where most of the value — and most of the remaining difficulty — actually sits.

The thing that makes it a decision rather than a ranking is cost. With a treatment costing $c = 0.4$ and a true average effect of 0.25, the arithmetic is brutal:

True average effect 0.25, treatment cost 0.4. The average is positive and below cost.

Treating everyone destroys value. Not "helps less than hoped" — the value gain is negative, because the average effect is below the cost of delivering it. On this programme, an evaluation reporting a positive average treatment effect and recommending rollout would be recommending a loss. The right question was never does it work on average but for whom does it beat the cost, and the oracle that treats exactly where $\tau(x) > c$ turns the same programme into +0.238 by enrolling 44% of people.

Between those two extremes sits the targeting curve: rank everyone by estimated effect, treat the top fraction, and plot realised value against that fraction. It climbs while high-effect units are being added, peaks, and falls as low-effect units dilute the gain. The peak lands at 0.237 at 44% treated — effectively the oracle's 0.238 — and the area over the random-targeting line, the Qini coefficient, is 0.206. That number is the honest valuation of the CATE model: not how accurate it is, but how much money the ranking is worth.

A Rule Someone Can Administer

A ranking is still not a rule, though. “Treat the top 44% by predicted effect” requires running a forest on every applicant and cannot be written into an eligibility document or audited by anyone. A policy tree fixes that: fit a shallow tree directly on the cost-adjusted, doubly-robust reward of treating each unit. That reward is a per-unit score combining an outcome model with a propensity model, so that either one being right suffices — and because the tree is fit on it directly, the tree optimises value rather than prediction accuracy, and what comes out is something a committee can read.

At depth 2 it recovers +0.238 — the oracle value — while treating 42%. The rule it learns is a single threshold on one covariate.

And it is worth comparing that threshold to the optimum rather than admiring the value. Since $\tau(x) = 2.5x_1 - 1.0$ and $c = 0.4$, the value-maximising boundary is exactly $x_1 > 0.560$. The tree settles on 0.579 — conservative by 0.019, treating slightly too few people, and giving up nothing measurable in value. The value surface is flat near its peak, which is why an approximate rule can be worth as much as an exact one, and is the reason interpretable policies cost less than intuition suggests.

One caution the notebook makes explicit. This tree is fit on doubly-robust scores derived from the same estimated $\hat\tau(x)$ whose quality the previous pages spent their length questioning, and its value is evaluated against the known truth — a luxury no real programme has. What transfers is the machinery and the shape of the answer, not the reassurance: a policy is only as good as the CATE estimates underneath it, and this page's clean result rests on the forest having got the ranking right, which it did even where it got the level wrong.

Where this sits

This closes the loop the causal-forest page opened: there, targeting the top 20% doubled the per-enrollee gain, and here that instinct becomes an optimisation with a cost in it. The doubly-robust scores are the AIPW construction reused as a reward, and the Qini curve is the causal cousin of a lift curve from the machine-learning arc — ranking quality valued in the units of the decision rather than in error.

Notebooks

References

• Athey, S. & Wager, S. (2021). Policy Learning With Observational Data. Econometrica 89(1), 133–161. — policy learning from observational data
• Kitagawa, T. & Tetenov, A. (2018). Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice. Econometrica 86(2), 591–616. — empirical welfare maximisation
• Manski, C. F. (2004). Statistical Treatment Rules for Heterogeneous Populations. Econometrica 72(4), 1221–1246. — statistical treatment rules