External Validity

Definition

External validity is the property that a causal estimate identified in one setting or subpopulation can be generalised to a broader population or a different setting. In the potential-outcomes framework, an estimate has external validity if the treatment effect is homogeneous across the relevant dimensions of heterogeneity — compliance type, covariate values, or time period.

Key Ideas

• In Fuzzy Regression Discontinuity (FRD): External validity requires compliance type $G_i$ to be independent of potential outcomes $(Y_i(0), Y_i(1))$ conditional on the forcing variable $X_i$. This ensures the Local Average Treatment Effect (LATE) for compliers at the threshold equals the Average Treatment Effect (ATE) for the full population at that $X$ value (Bertanha-Imbens 2014).
• Testable implication: External validity implies $E[Y^{\text{obs}} \mid W=w, X]$ is continuous in $X$ at the threshold $T^*$ for each $w \in \{0,1\}$. A discontinuity in either treatment-arm-specific conditional expectation constitutes evidence against external validity.
• Comparison to Hausman/Angrist: The Bertanha-Imbens pair of restrictions is strictly stronger than either the Hausman test or the Angrist (2004) test individually, and is transformation-invariant (the Angrist restriction can hold for Y but fail for log Y).
• Identification of ATE: Under external validity in FRD, $\tau^{\text{ATE}}$ is identified over the full support of $X$ from observed treated and non-treated outcomes, not just at the threshold.

Why It Matters

Most quasi-experimental and instrumental variable (IV) estimates are local — they apply to the compliers who are shifted by the instrument, at the particular covariate values near the discontinuity or manipulation. External validity is the additional condition needed to generalise to policymakers' actual population of interest. Knowing when external validity fails (and which compliance groups differ most) helps bound the gap between LATE and ATE.

Open Questions

• Extrapolation methods when external validity fails: derivative extrapolation (Dong-Lewbel 2014), covariate conditioning (Angrist-Fernandez-Val 2010), running-variable bandwidth (Angrist-Rokkanen 2012).
• External validity in repeated cross-sections vs. panel data.
• When to prefer ATE vs. LATE as the policy-relevant estimand (Heckman-Vytlacil 2005).

Related

• Fuzzy Regression Discontinuity
• Local Average Treatment Effect
• Bertanha-Imbens (2014)