Double Machine Learning

Definition

Double (or debiased) machine learning (DML; Chernozhukov et al. 2018) is a framework for estimating a low-dimensional causal parameter $\theta_0$ — a treatment effect, an elasticity, a structural coefficient — while using flexible machine-learning methods to estimate high-dimensional nuisance functions (e.g. the outcome regression and the propensity/treatment model), and still obtaining $\sqrt{n}$-consistent, asymptotically normal inference with honest standard errors. It rests on three ingredients: a Neyman-orthogonal score, high-quality ML nuisance estimation, and cross-fitting (Bach et al. 2023).

Key Ideas

• The regularization-bias problem. Plugging ML estimates of the nuisance functions into a naive moment condition transmits the ML regularization/overfitting bias directly into $\hat\theta_0$, so the naive estimator is biased and not $\sqrt{n}$-consistent.
• Neyman orthogonality. Use a score $\psi(W;\theta,\eta)$ whose expectation has zero derivative with respect to the nuisance $\eta$ at the truth (a Gateaux derivative condition). First-order errors in $\hat\eta$ then have only a second-order effect on $\hat\theta_0$ — the estimator is locally insensitive to nuisance estimation error. (For the partially linear model this means partialling the treatment and outcome on the covariates and using the residual-on-residual moment.)
• Cross-fitting (sample splitting). Estimate the nuisances on one part of the data and evaluate the orthogonal score on the held-out part, averaging over folds. This removes the own-observation overfitting term, so no Donsker/complexity restriction on the ML learners is needed.
• Rate condition. Orthogonality tolerates slow nuisance convergence, but the product of the nuisance error rates must be $o(n^{-1/2})$ (each roughly $o(n^{-1/4})$) for the target to be $\sqrt{n}$-normal.
• Model menu. Partially linear regression (PLR), the interactive regression model (IRM) for ATE/ATTE with a fully nonparametric treatment interaction, and instrumental-variable versions (PLIV, IIVM).

How It Works

For the partially linear model $Y = \theta_0 D + g_0(X) + \varepsilon$, $D = m_0(X) + V$: (1) estimate $g_0$ and $m_0$ by any ML learner on a training fold; (2) form residuals $\tilde Y = Y - \hat g_0(X)$ and $\tilde D = D - \hat m_0(X)$ on the held-out fold; (3) estimate $\theta_0$ from the orthogonal moment $E[(\tilde Y - \theta \tilde D)\tilde D]=0$, i.e. an IV/residual-regression of $\tilde Y$ on $\tilde D$; (4) repeat across folds and average. The variance follows from the influence function of the orthogonal score, giving standard confidence intervals.

Why It Matters

• Flexible controls with valid inference. It reconciles two goals usually in tension — using expressive ML for confounding adjustment, and reporting honest confidence intervals for a causal effect.
• General and modular. Any sufficiently accurate learner (lasso, random forest, boosting, neural nets) can supply the nuisances; the same orthogonal-score + cross-fitting recipe covers many causal models — implemented in the R/Python DoubleML packages.
• Bridges ML and econometrics. It is a central tool in the modern causal-machine-learning literature, connecting tree/forest predictors and IV/LATE estimands.

Open Questions

• Finite-sample performance. Guarantees are asymptotic; the $o(n^{-1/4})$ nuisance-rate condition is hard to verify and small samples can still be biased.
• Learner and tuning choice. Results depend on choosing and tuning nuisance learners well; poor learners break the rate condition.
• Beyond low-dimensional targets. Extending honest inference to many or high-dimensional target parameters (simultaneous confidence bands) is more delicate.

Related

• Machine Learning in Asset Pricing
• Local Average Treatment Effect
• Classification and Regression Trees (CART)
• Bach et al. (2023) DoubleML — An Object-Oriented Implementation of Double Machine Learning in R
• Victor Chernozhukov
• Philipp Bach