Difference-in-Differences

Definition

Difference-in-differences (DiD) is a causal identification strategy that compares the change in outcomes over time for a group affected by a treatment to the change in outcomes over time for an unaffected comparison group. By subtracting each group's time trend from the other's, DiD removes both time-invariant group differences and common time shocks, isolating the treatment effect — provided the treatment and comparison groups would have evolved in parallel absent the treatment (the parallel trends assumption).

Key Ideas

Estimand: $\hat{\beta}_{\text{DiD}} = (\bar{Y}_1^1 - \bar{Y}_0^1) - (\bar{Y}_1^0 - \bar{Y}_0^0)$ , where superscripts denote group (treatment $= 1$ , control $= 0$ ) and subscripts denote period (pre $= 0$ , post $= 1$ ). The first difference removes the common time trend $\alpha_1$ ; the second difference removes the permanent group mean difference $\alpha^1$ .
Parallel trends assumption: The identifying assumption is that in the absence of treatment, $\beta = 0$ — no omitted interaction between group membership ( $j = 1$ ) and the post-treatment period ( $t = 1$ ) other than the treatment itself. Written: $E[\varepsilon \mid d^*] = 0$ where $d^*$ is the treatment indicator. The parallel trends assumption is untestable for the post-treatment period; examining pre-treatment trends is the standard diagnostic.
Regression formulation: Ordinary Least Squares (OLS) on $y_{it}^j = \alpha + \alpha_1 d_t + \alpha^1 d_j + \beta d_i^* + \varepsilon_{it}$ reproduces $\hat{\beta}_{\text{DiD}}$ and makes the key assumption explicit as a zero restriction on the group $\times$ time interaction beyond $d^*$ . Adding covariates $z_{it}$ adjusts for observable differences and reduces residual variance; requires equal coefficient vectors across groups.
Error structure and serial correlation: In repeated cross-sections, $\varepsilon_{i0}$ and $\varepsilon_{i1}$ are independent. In panel data, within-unit correlation reduces variance of $\hat{\beta}$ . Group-level error terms (clustering at the group×period level) must be accounted for to avoid understated standard errors (Misspecified variances threat in Campbell's framework). Serial correlation is the dominant inferential threat in multi-period DiD. Bertrand, Duflo, and Mullainathan (2003) show that when outcomes are serially correlated ( $\rho_1 \approx 0.5$ for wages; similarly high for employment, health) and time series are long (average 16.5 periods in their survey of 92 DD papers), OLS standard errors produce false-rejection rates of 44–67% on placebo laws rather than the nominal 5%. A survey of 92 DD papers (1990–2000) finds only 5/92 correct for serial correlation. The fix: cluster standard errors at the group level (e.g., the entire state, not the state×year cell) using the arbitrary variance-covariance estimator — implemented as cluster(state) in Stata. This works well for $N \geq 50$ groups but degrades for small $N$ . Alternatively, collapse all pre-treatment years to one pre-period and all post-treatment years to one post-period per group (pre/post aggregation) — this always works but sacrifices power.
Studies without a time dimension: The DiD logic applies whenever two binary dimensions define a treatment through their interaction; t need not indicate time. Madrian (1994) uses insurance-demand level × current coverage status to identify job-lock from employment-based health insurance.

Triple-Differences

When treatment varies along three dimensions, the treatment indicator is a three-way interaction:

$y_{it}^{jk} = \alpha + \alpha_1 d_t + \alpha^1 d_j + \gamma^1 e_k + \alpha_1^1 d_t^j + \gamma_1^1 e_t^k + \alpha^{11} d^{jk} + \beta d_t^{jk} + \varepsilon_{it}^{jk}$

The coefficient $\beta$ on the three-way interaction $d_t^{jk}$ is the treatment effect. Including all main effects and first-order interactions is required; omitting them confounds the three-way effect with the omitted lower-order interactions.

Examples:

Gruber (1994): treated = women of certain ages (k=1) × certain states (j=1) × post-mandate (t=1). β identifies the incidence of mandated maternity benefits.
Yelowitz (1994): treated = mothers with children of certain ages (k=1) × certain states (j=1) × post-Medicaid expansion (t=1). β identifies the effect of Medicaid expansion on welfare participation and labor supply.

Triple-differences removes state×time and state×group trends simultaneously, exploiting the demographic-group variation within states as the identifying variation.

Probing Comparability

The DiD design rests on the comparability of treatment and comparison groups conditional on removing common time trends. Several design features test this assumption:

Multiple comparison groups: Additional control groups test overidentifying restrictions — if the treatment has no effect on a similar group, the parallel trends assumption is more credible. Formally equivalent to Rosenbaum's "control by systematic variation." Multiple comparison groups with different potential biases provide bounds on the estimate.
Multiple pre/post time periods: A long pre-treatment time series allows examination of whether treatment and control groups move in parallel before treatment. Reveals seasonality, secular differential trends, and group effects in the error term. Meyer (1992): quarterly Unemployment Insurance (UI) claim data reveals strong seasonality — comparing only the months immediately before and after a benefit increase would be invalid.
Treatment reversal: If the policy is later repealed, the outcome should reverse, providing additional evidence of causality (Curington 1994).
Positive treatment group: Examine whether the design can detect a known causal effect (Margolin 1987).

Area FE vs. Individual FE vs. Population-Segment FE (Moffitt 2005)

DiD designs come in three distinct variants that differ in what "groups" and "changes" refer to:

Area fixed-effects model: Geographic areas are the groups; ΔZ (change in area-level variable) instruments for ΔT. Individual changes in T are presumed endogenous even after differencing. This is the fully identified design: two equations (outcome and selection). See Instrumental Variables and Natural Experiments.
Individual fixed-effects model: The design that only estimates ΔY = α + βΔT + γΔX + ε without an instrument for ΔT. Assumes differencing eliminates all bias from unobservables. Moffitt (2005) notes this model "has seen decreasing support among economists" because it leaves unexplained why individual changes in T occur — those changes may be endogenous.
Population-segment fixed-effects (Moffitt's term for demographic DiD): Nationwide policies affecting demographic groups differently (e.g., single mothers vs. childless women). Mathematically equivalent to area FE but with demographic groups replacing geographic areas. The parallel-trends assumption is "much more suspect" for demographic groups, which differ on many time-varying observables and unobservables, than for geographic areas.

The key practical implication: DiD designs that do not include an instrument for the within-group variation in T (i.e., individual or population-segment fixed-effects designs) are weaker than area fixed-effects designs that explicitly instrument for individual T changes.

Connection to Instrumental Variables

A policy-change dummy variable in a DiD design can serve directly as an instrument for a continuous endogenous treatment variable. When the policy deterministically assigns treatment groups to a changed benefit level, the policy dummy becomes the first-stage instrument in Two-Stage Least Squares (2SLS), and the second-stage estimates the derivative of the outcome with respect to the continuous benefit amount rather than the binary treatment indicator. This is the connection between DiD and Instrumental Variables in the workers' compensation literature (Meyer, Viscusi, and Durbin 1990/1995).

Internal Validity Threats Addressed by DiD

DiD addresses several of Campbell's internal validity threats relative to simple before/after:

Omitted variables: Removed if common to treatment and comparison groups
Trends in outcomes: Removed if common to both groups (parallel trends)
Mismeasurement: Removed if changes in survey methods affect both groups equally
Attrition: Reduced relative to single-group designs; examine whether samples are selected similarly over time

The main residual threat is the omitted interaction (Campbell threat 9) — a differential trend or differential response to an omitted variable that affects one group but not the other. This is precisely what the parallel trends assumption rules out and what pre-trend tests probe.

Open Questions

How much pre-treatment parallel movement is "enough" to make parallel trends credible post-treatment?
When treatment and comparison groups have very different baseline levels, nonlinear functional forms may yield different DiD estimates — which specification is correct?
The clustering level for standard errors (individual, group, group×period) is not always obvious and materially affects inference. Bertrand et al. (2003) resolve this for the canonical multi-period DiD: cluster at the group (state) level, not the group×period level.
Treatment effect heterogeneity across groups and timing in staggered adoption designs (Callaway and Sant'Anna 2021; de Chaisemartin and D'Haultfoeuille 2020) creates additional complications beyond the serial correlation problem.