Causal Inference VII(b) — Synthetic Difference-in-Differences

Unifying synthetic control and DiD: unit and time weights (Arkhangelsky et al. 2021)

The synthetic-control notebook built a weighted donor pool to mimic California before Proposition 99; the difference-in-differences notebook took a double difference against an equal-weighted control group. Each fixes one of the other's weaknesses and keeps its own. Synthetic difference-in-differences (SDID) (Arkhangelsky, Athey, Hirshberg, Imbens & Wager, 2021) combines them, and dominates both:

• DiD uses equal unit weights (all controls count the same) and equal time weights, and relies on parallel trends — which fail when the control group is nothing like the treated unit.
• Synthetic control optimizes unit weights to match the treated unit's pre-trend, but demands a near-exact pre-treatment fit and weights all pre-periods equally.
• Synthetic DiD optimizes both unit weights (like SC, but with an intercept that allows a level shift — so it does not need an exact fit) and time weights (emphasizing the pre-periods most predictive of the post-period), then computes a doubly-weighted double-difference. Relaxing SC's exact-fit and DiD's parallel-trends at once makes it more robust than either.

The SDID estimator is a weighted DiD: $$\hat\tau^{sdid}=\Big(\bar Y^{tr}_{post}-\sum_t\lambda_t Y^{tr}_t\Big)-\sum_i\omega_i\Big(Y^{i}_{post}-\sum_t\lambda_t Y^{i}_t\Big),$$ with unit weights $\omega_i$ and time weights $\lambda_t$ chosen by regularized pre-fit problems. We build it from scratch on the Proposition 99 data and compare all three estimators; the from-scratch SDID reproduces Arkhangelsky et al.'s published estimate. Python-lead (from-scratch); R companion mirrors it (with Synth and fixest for the SC and DiD comparisons).

1. The three estimators on Proposition 99

Using the same California tobacco panel (39 states, 1970–2000), we compute all three effects. DiD — California minus the equal-weighted average of all donor states — is badly biased, because the average donor state is nothing like California (higher, differently-trending smoking). Synthetic control optimizes unit weights to match California's pre-1988 path, giving a smaller, more credible effect. Synthetic DiD optimizes unit weights (with a level-shifting intercept and ridge regularization) and time weights, and lands between/beyond them at the Arkhangelsky et al. estimate.

import numpy as np, pandas as pd, matplotlib.pyplot as plt, warnings
warnings.filterwarnings("ignore")
from scipy.optimize import minimize
BLUE="#2b6cb0"; RED="#c53030"; GREEN="#2f855a"; ORANGE="#dd6b20"; GREY="#a0aec0"; PURP="#6b46c1"
d=pd.read_csv("smoking.csv"); piv=d.pivot(index="year",columns="state",values="cigsale"); yrs=piv.index.values
treated="California"; donors=[c for c in piv.columns if c!=treated]; pre=yrs<=1988; post=yrs>=1989
Y1=piv[treated].values; Y0=piv[donors].values; Npre=int(pre.sum()); Npost=int(post.sum())
did=(Y1[post].mean()-Y1[pre].mean())-(Y0[post].mean()-Y0[pre].mean())
def sc_weights(A,b):
    n=A.shape[1]; res=minimize(lambda w:((A@w-b)**2).sum(),np.ones(n)/n,method="SLSQP",bounds=[(0,1)]*n,
        constraints=[{"type":"eq","fun":lambda w:w.sum()-1}],options={"maxiter":2000,"ftol":1e-12}); return res.x
w_sc=sc_weights(Y0[pre],Y1[pre]); sc=(Y1[post]-Y0[post]@w_sc).mean()
# --- SDID ---
zeta=(Npost)**0.25*np.std(np.diff(Y0[pre],axis=0))                    # ridge penalty (Arkhangelsky et al.)
def unit_weights(Y0pre,Y1pre):
    n=Y0pre.shape[1]
    def obj(p): w0,w=p[0],p[1:]; r=w0+Y0pre@w-Y1pre; return r@r+zeta**2*Npre*(w@w)
    res=minimize(obj,np.r_[0,np.ones(n)/n],method="SLSQP",bounds=[(None,None)]+[(0,1)]*n,
        constraints=[{"type":"eq","fun":lambda p:p[1:].sum()-1}],options={"maxiter":3000,"ftol":1e-12}); return res.x[1:]
def time_weights(Y0pre,Y0post):
    m=Y0pre.shape[0]; target=Y0post.mean(0)
    def obj(p): l0,l=p[0],p[1:]; r=l0+l@Y0pre-target; return r@r
    res=minimize(obj,np.r_[0,np.ones(m)/m],method="SLSQP",bounds=[(None,None)]+[(0,1)]*m,
        constraints=[{"type":"eq","fun":lambda p:p[1:].sum()-1}],options={"maxiter":3000,"ftol":1e-12}); return res.x[1:]
om=unit_weights(Y0[pre],Y1[pre]); lam=time_weights(Y0[pre],Y0[post])
sdid=(Y1[post].mean()-lam@Y1[pre])-(Y0[post].mean(0)@om-lam@(Y0[pre]@om))
print("Proposition 99 effect (per-capita cigarette packs):")
print(f"  DiD (equal weights)   = {did:.2f}   (biased: donor average unlike California)")
print(f"  Synthetic Control     = {sc:.2f}")
print(f"  Synthetic DiD (SDID)  = {sdid:.2f}   (Arkhangelsky et al. report ~ -15.6)")
fig,ax=plt.subplots(figsize=(8,4))
nm=["DiD","Synthetic\nControl","Synthetic DiD"]; vals=[did,sc,sdid]
ax.bar(nm,vals,color=[GREY,BLUE,GREEN]); ax.axhline(0,color="k",lw=.7)
for i,v in enumerate(vals): ax.text(i,v-1.5,f"{v:.1f}",ha="center",color="white",fontweight="bold")
ax.set_ylabel("estimated effect (packs per capita)"); ax.set_title("Prop 99: DiD vs Synthetic Control vs Synthetic DiD")
plt.tight_layout(); plt.show()
print(f"DiD ({did:.1f}) overstates the effect (equal-weighted controls mis-trend); SC ({sc:.1f}) and SDID ({sdid:.1f}) are more")
print("credible. From-scratch SDID matches the published Arkhangelsky et al. estimate of about -15.6 packs.")

Proposition 99 effect (per-capita cigarette packs):
  DiD (equal weights)   = -27.35   (biased: donor average unlike California)
  Synthetic Control     = -19.51
  Synthetic DiD (SDID)  = -15.60   (Arkhangelsky et al. report ~ -15.6)

DiD (-27.3) overstates the effect (equal-weighted controls mis-trend); SC (-19.5) and SDID (-15.6) are more
credible. From-scratch SDID matches the published Arkhangelsky et al. estimate of about -15.6 packs.

2. The two sets of weights that define SDID

SDID's robustness comes from its two weight vectors, which we inspect. The unit weights $\omega_i$ (like synthetic control, but regularized and with a level-shifting intercept) pick a sparse set of donor states whose trend — not necessarily level — matches California's. The time weights $\lambda_t$ are SDID's distinctive addition: they up-weight the pre-treatment years most predictive of the post-treatment period, discounting distant history that is less relevant. Because the intercept absorbs any constant California-vs-synthetic level gap, SDID does not need the exact pre-period fit that synthetic control requires — it corrects a level difference with the "difference" part of DiD and a trend difference with the "synthetic" part.

fig,ax=plt.subplots(1,2,figsize=(13,4.4))
top=pd.Series(om,index=donors).sort_values(ascending=False).head(8)
ax[0].barh(top.index[::-1],top.values[::-1],color=GREEN); ax[0].set_xlabel("unit weight ωᵢ"); ax[0].set_title("SDID unit weights (top donor states)")
preyrs=yrs[pre]
ax[1].bar(preyrs,lam,color=PURP); ax[1].set_xlabel("pre-treatment year"); ax[1].set_ylabel("time weight λₜ")
ax[1].set_title("SDID time weights: recent pre-years matter most")
plt.tight_layout(); plt.show()
print(f"Unit weights concentrate on a few states (top: {', '.join(top.index[:4])}); time weights emphasize the late-1980s,")
print("the pre-years most predictive of the post-1988 trajectory. These two reweightings are what make SDID robust.")

Unit weights concentrate on a few states (top: Nevada, New Hampshire, Connecticut, Delaware); time weights emphasize the late-1980s,
the pre-years most predictive of the post-1988 trajectory. These two reweightings are what make SDID robust.

3. Inference and robustness

SDID admits a straightforward jackknife standard error — leave out one control state at a time, recompute, and use the spread of the estimates. We report it, and then the estimator Arkhangelsky et al. actually prescribe here.

The jackknife variance estimator requires several treated units and is not valid with one. Proposition 99 has exactly one: California. For that case the authors prescribe the placebo estimator — hand the treatment label to each donor in turn and take the spread of the resulting estimates — and it is what their own Prop 99 application uses.

The difference is not cosmetic. The placebo standard error comes out at 9.49 against the jackknife's 2.37, four times larger, and the interval it produces includes zero. The estimate of about −15.6 packs is solid and reproduces the published figure; the claim that it is clearly distinguishable from zero does not survive. That also reconciles this page with the synthetic-control one, which found $p = 0.077$ on the same data. SDID's robustness is not just empirical: Arkhangelsky et al. prove it is consistent and asymptotically normal under weaker conditions than either SC or DiD — it does not require the near-perfect pre-fit synthetic control needs, nor the parallel trends DiD assumes, because the two reweightings jointly absorb level and trend mismatches. In practice it is a sensible default for comparative-case-study / panel policy evaluation.

def sdid_estimate(Y0m):                                   # SDID on a given donor matrix (for jackknife)
    zt=(Npost)**0.25*np.std(np.diff(Y0m[pre],axis=0))
    n=Y0m.shape[1]
    def uw(p): w0,w=p[0],p[1:]; r=w0+Y0m[pre]@w-Y1[pre]; return r@r+zt**2*Npre*(w@w)
    o=minimize(uw,np.r_[0,np.ones(n)/n],method="SLSQP",bounds=[(None,None)]+[(0,1)]*n,constraints=[{"type":"eq","fun":lambda p:p[1:].sum()-1}],options={"maxiter":2000}).x[1:]
    m=Y0m[pre].shape[0]; tgt=Y0m[post].mean(0)
    def tw(p): l0,l=p[0],p[1:]; r=l0+l@Y0m[pre]-tgt; return r@r
    l=minimize(tw,np.r_[0,np.ones(m)/m],method="SLSQP",bounds=[(None,None)]+[(0,1)]*m,constraints=[{"type":"eq","fun":lambda p:p[1:].sum()-1}],options={"maxiter":2000}).x[1:]
    return (Y1[post].mean()-l@Y1[pre])-(Y0m[post].mean(0)@o-l@(Y0m[pre]@o))
jk=np.array([sdid_estimate(np.delete(Y0,j,axis=1)) for j in range(Y0.shape[1])])
Nc=Y0.shape[1]; se=np.sqrt((Nc-1)/Nc*np.sum((jk-jk.mean())**2))
print(f"Synthetic DiD estimate = {sdid:.2f} packs")
print(f"jackknife SE = {se:.2f};  95% CI = [{sdid-1.96*se:.2f}, {sdid+1.96*se:.2f}]")
print()
# Arkhangelsky et al. prescribe the PLACEBO estimator when there is ONE treated unit;
# the jackknife requires several. California is the only treated unit here.
def sdid_generic(Y0m, y1):
    zt=(Npost)**0.25*np.std(np.diff(Y0m[pre],axis=0)); nn=Y0m.shape[1]
    def uw(q): w0,w=q[0],q[1:]; r=w0+Y0m[pre]@w-y1[pre]; return r@r+zt**2*Npre*(w@w)
    o=minimize(uw,np.r_[0,np.ones(nn)/nn],method="SLSQP",bounds=[(None,None)]+[(0,1)]*nn,
               constraints=[{"type":"eq","fun":lambda q:q[1:].sum()-1}],options={"maxiter":2000}).x[1:]
    mm=Y0m[pre].shape[0]; tgt=Y0m[post].mean(0)
    def tw(q): l0,l=q[0],q[1:]; r=l0+l@Y0m[pre]-tgt; return r@r
    lz=minimize(tw,np.r_[0,np.ones(mm)/mm],method="SLSQP",bounds=[(None,None)]+[(0,1)]*mm,
                constraints=[{"type":"eq","fun":lambda q:q[1:].sum()-1}],options={"maxiter":2000}).x[1:]
    return (y1[post].mean()-lz@y1[pre])-(Y0m[post].mean(0)@o-lz@(Y0m[pre]@o))
plac=np.array([sdid_generic(np.delete(Y0,j,axis=1), Y0[:,j]) for j in range(Y0.shape[1])])
se_pl=plac.std(ddof=1); more=int((np.abs(plac)>=abs(sdid)).sum()); p_pl=(more+1)/(len(plac)+1)
print(f"placebo   SE = {se_pl:.2f};  95% CI = [{sdid-1.96*se_pl:.2f}, {sdid+1.96*se_pl:.2f}]")
print(f"  placebo spread over {len(plac)} donors: mean {plac.mean():+.2f}, min {plac.min():.1f}, max {plac.max():.1f}")
print(f"  placebos at least as extreme as California: {more} of {len(plac)}  ->  p = {p_pl:.3f}")
print()
print("The two disagree, and the wider one is the one to believe. Arkhangelsky et al. show the jackknife")
print("variance estimator requires SEVERAL treated units and is not valid with one; for a single treated")
print("unit they prescribe the placebo estimator, and that is what their own Proposition 99 application")
print("uses. California is the only treated unit here.")
print()
print(f"The correction matters. The placebo standard error is {se_pl/se:.1f}x the jackknife, and the interval it")
print(f"gives INCLUDES ZERO. On this evidence the effect is not significant at 5%; the permutation p of")
print(f"{p_pl:.3f} sits right at the boundary.")
print()
print("That reconciles this page with the synthetic-control one, which found p = 0.077 on the same data")
print("and noted that a single treated unit admits no conventional standard error. The estimate of about")
print("-15.6 packs is solid and reproduces the published figure; what does not survive is the claim that")
print("it is clearly distinguishable from zero. One treated state and 38 donors cannot settle that.")
# effect trajectory: treated minus synthetic-DiD counterfactual over time
cf=lam@Y1[pre] - (Y0[pre]@om)@lam + Y0.mean(0)*0   # not needed; show gap series
synth_path=Y0@om + (lam@Y1[pre] - lam@(Y0[pre]@om))   # SDID counterfactual for CA (level-shifted synthetic)
fig,ax=plt.subplots(figsize=(9,4.6))
ax.plot(yrs,Y1,color=RED,lw=2.5,label="California (actual)")
ax.plot(yrs,synth_path,color=GREEN,lw=2.5,ls="--",label="synthetic-DiD counterfactual")
ax.axvline(1988,color="k",ls=":"); ax.set_xlabel("year"); ax.set_ylabel("cigarette sales (packs)")
ax.set_title(f"Synthetic DiD: California vs its counterfactual (effect {sdid:.1f}, placebo 95% ± {1.96*se_pl:.1f})"); ax.legend()
plt.tight_layout(); plt.show()
print("The level-shift intercept lets the synthetic-DiD counterfactual sit parallel to California pre-1988")
print("without matching it exactly; the post-1988 gap is the estimated effect.")

fig,axes=plt.subplots(1,2,figsize=(11,4.2))
ax=axes[0]
ax.hist(plac,bins=14,color="#94a3b8",edgecolor="white",alpha=.9)
ax.axvline(sdid,color="#c53030",lw=2.5,label=f"California {sdid:.1f}")
ax.axvline(0,color="#475569",lw=1,ls=":")
ax.set_xlabel("SDID estimate when the donor is labelled treated")
ax.set_ylabel("donor states")
ax.set_title(f"Placebo distribution: {more} of {len(plac)} donors at least as extreme  (p = {p_pl:.3f})")
ax.legend(fontsize=8)

ax=axes[1]
for i,(lab,s,col) in enumerate([("jackknife\n(needs several treated units)",se,"#94a3b8"),
                                ("placebo\n(prescribed for one)",se_pl,"#c53030")]):
    ax.errorbar(sdid,i,xerr=1.96*s,fmt="o",color=col,lw=3,capsize=6,markersize=7)
    ax.text(sdid-1.96*s,i+.22,f"[{sdid-1.96*s:.1f}, {sdid+1.96*s:.1f}]",fontsize=8,color=col)
ax.axvline(0,color="#475569",lw=1.5,ls="--")
ax.set_yticks([0,1]); ax.set_yticklabels(["jackknife\n(needs several treated units)",
                                          "placebo\n(prescribed for one)"],fontsize=8)
ax.set_ylim(-.6,1.6); ax.set_xlabel("effect on cigarette sales (packs per capita)")
ax.set_title("The interval that applies here includes zero")
plt.tight_layout(); plt.show()
print("Left: California sits inside the placebo spread rather than outside it -- one donor state produces")
print("an estimate at least as large without any policy change. Right: the jackknife interval clears zero")
print("and the placebo interval does not, and only the second one is valid with a single treated unit.")

Synthetic DiD estimate = -15.60 packs
jackknife SE = 2.37;  95% CI = [-20.25, -10.96]

placebo   SE = 9.49;  95% CI = [-34.21, 3.01]
  placebo spread over 38 donors: mean +0.39, min -31.8, max 14.9
  placebos at least as extreme as California: 1 of 38  ->  p = 0.051

The two disagree, and the wider one is the one to believe. Arkhangelsky et al. show the jackknife
variance estimator requires SEVERAL treated units and is not valid with one; for a single treated
unit they prescribe the placebo estimator, and that is what their own Proposition 99 application
uses. California is the only treated unit here.

The correction matters. The placebo standard error is 4.0x the jackknife, and the interval it
gives INCLUDES ZERO. On this evidence the effect is not significant at 5%; the permutation p of
0.051 sits right at the boundary.

That reconciles this page with the synthetic-control one, which found p = 0.077 on the same data
and noted that a single treated unit admits no conventional standard error. The estimate of about
-15.6 packs is solid and reproduces the published figure; what does not survive is the claim that
it is clearly distinguishable from zero. One treated state and 38 donors cannot settle that.

The level-shift intercept lets the synthetic-DiD counterfactual sit parallel to California pre-1988
without matching it exactly; the post-1988 gap is the estimated effect.

Left: California sits inside the placebo spread rather than outside it -- one donor state produces
an estimate at least as large without any policy change. Right: the jackknife interval clears zero
and the placebo interval does not, and only the second one is valid with a single treated unit.

4. Summary

Synthetic difference-in-differences unifies the two workhorses of comparative-case-study evaluation. It optimizes unit weights (like synthetic control, so the comparison group tracks the treated unit's trend) and time weights (emphasizing the most relevant pre-periods), and — via a level-shifting intercept — takes a double difference (like DiD, so it tolerates a constant level gap rather than demanding an exact pre-fit). On Proposition 99 the three estimators separated cleanly: DiD ($-27$, biased by mismatched controls), synthetic control ($-20$), and synthetic DiD ($-15.6$, matching Arkhangelsky et al.). Inference is the weaker half of the story: the jackknife standard error usually quoted is not valid with a single treated unit, and the placebo estimator the authors prescribe instead is four times larger and gives an interval that includes zero ($p = 0.051$). The point estimate is reproducible and credible; its distinguishability from zero is not established by this design.

SDID is more robust than either parent because its two reweightings jointly absorb the level and trend mismatches that break DiD (parallel trends) and SC (exact pre-fit), and it comes with formal asymptotic guarantees. Guidance: for panel policy evaluation with a treated unit (or units) and a donor pool, synthetic DiD is a strong default — report it alongside SC and DiD and check they agree. Cross-links: SDID is the synthesis of the Synthetic Control (subsection 7, unit weights) and Difference-in-Differences (subsection 6, double difference) notebooks; its doubly-robust flavour echoes the AIPW/DML/TMLE theme (two models/weightings, each covering the other's failure); the jackknife inference parallels the placebo/permutation inference of the synthetic-control notebook. This completes the depth of the Synthetic Control subsection.