Synthetic Difference-in-Differences

Python · from scratch  ·  R · synthdid  ·  Arkhangelsky et al. (2021), Proposition 99

Taking Both Sets of Weights

The two designs in this pair each fix the other's weakness and keep their own. Difference-in-differences uses equal unit weights and relies on parallel trends — that the two groups would have moved together absent treatment — which fails when the control group is nothing like the treated unit. Synthetic control optimises unit weights to match the pre-trend, but demands a near-exact pre-treatment fit and weights every pre-period equally.

Synthetic difference-in-differences (Arkhangelsky, Athey, Hirshberg, Imbens & Wager 2021) takes both. It optimises unit weights like synthetic control — but with a level-shifting intercept, so it needs to match California's trend rather than its level — and adds time weights that up-weight the pre-treatment years most predictive of the post-treatment period. Then it takes a doubly-weighted double difference:

τ^sdid=(Yˉposttr−∑tλtYttr)−∑iωi(Yposti−∑tλtYti)\hat\tau^{sdid}=\Big(\bar Y^{tr}_{post}-\sum_t\lambda_t Y^{tr}_t\Big)-\sum_i\omega_i\Big(Y^{i}_{post}-\sum_t\lambda_t Y^{i}_t\Big)

The two relaxations compose. The intercept absorbs a constant level gap, so the exact pre-fit synthetic control requires is unnecessary; the unit weights absorb a trend mismatch, so the parallel trends DiD assumes are not needed either.

On the same Proposition 99 panel the three estimators separate cleanly, and the from-scratch implementation lands on the published figure.

estimatoreffectwhat it assumes
difference-in-differences−27.35parallel trends against an equal-weighted average
synthetic control−19.51a near-exact pre-treatment fit
synthetic DiD−15.60neither — matches trend, absorbs level
published figure≈ −15.6Arkhangelsky et al. (2021)

The −15.60 reproducing Arkhangelsky et al.'s reported −15.6 is a genuine cross-check on the implementation, not a coincidence of rounding. The ordering also makes sense: DiD is most biased because its equal-weighted donor average both sits above California and trends differently, and each reweighting pulls the estimate toward something more defensible.

The two weight vectors are worth inspecting separately. Unit weights concentrate on a handful of states — Nevada, New Hampshire, Connecticut, Delaware — chosen for trend rather than level, since the intercept handles the rest. Time weights emphasise the late 1980s, the pre-treatment years most predictive of the post-1988 trajectory, discounting distant history that carries less information about where California was heading.

The Standard Error That Does Not Apply

Then the inference, where the notebook had it wrong and the correction changes the conclusion. SDID admits a jackknife standard error — drop one control state, recompute, take the spread — and that gives 2.37, an interval comfortably clear of zero.

But the jackknife variance estimator requires several treated units and is not valid with one. Proposition 99 has exactly one: California. For that case Arkhangelsky et al. prescribe the placebo estimator — hand the treatment label to each donor in turn and take the spread of the resulting estimates — and it is what their own Prop 99 application uses.

variance estimatorSE95% intervalapplicable here?
jackknife2.37[−20.25, −10.96]no — needs several treated units
placebo9.49[−34.21, +3.01]yes — prescribed for one

Placebo distribution over 38 donors: mean +0.39, range −31.8 to +14.9. One donor is at least as extreme as California, giving p = 0.051.

The placebo standard error is 9.49, four times the jackknife, and the interval it produces includes zero. One of 38 placebo states is at least as extreme as California, giving p=0.051p = 0.051 — right at the boundary.

So the estimate of about −15.6 packs is solid and reproduces the published figure, while its distinguishability from zero is not established by this design. That also reconciles this page with the previous one, which found p=0.077p = 0.077 on the same data and observed that a single treated unit admits no conventional standard error. Two estimators, two inference procedures, and the same honest verdict: one treated state and 38 donors cannot settle significance, however good the point estimate.

Which is the right note for this group to end on. Each design here borrows identification from the world — an instrument, a threshold, a policy date, a comparison region — and each pays for it in a currency the headline number does not show. Here the currency is degrees of freedom for inference: with one treated unit there is very little to compute a standard error from, and choosing an estimator that appears to produce one anyway is the failure mode.

Where this sits

SDID is the synthesis of Synthetic Control's unit weights and Difference-in-Differences' double difference. Its doubly-robust flavour — two reweightings, each covering the other's failure — echoes the AIPW and TMLE estimators in modern balancing. And its placebo inference is the permutation logic of Fisher's exact test, which is where this whole arc began.

Notebooks

Python Synthetic DiD built from scratch on the Proposition 99 panel: ridge-regularised unit weights with a level-shifting intercept, time weights over the pre-treatment years, the doubly-weighted double difference reproducing the published −15.6, and both the jackknife and the placebo variance estimators with the reason the second is the applicable one (Python) Open → R The R companion using the reference synthdid implementation from the authors of the method, with its own weight diagnostics and variance options (R) Open →

Downloads

smoking.csv The Proposition 99 panel — 39 states, 1970–2000 — the same data the synthetic-control page uses, so the three estimators are directly comparable

References