Causal Inference VI(b) — Honest DiD (R companion)¶
HonestDiD — relative-magnitudes and smoothness sensitivity for parallel-trends violations¶
HonestDiD (Rambachan & Roth) is the reference implementation. Taking the event-study estimates and covariance from the Python notebook (same robust and fragile scenarios), it computes the robust confidence sets and the breakdown value $\bar M^\star$ under both the relative-magnitudes and smoothness restrictions, and draws the sensitivity plots. Results match the Python notebook: robust $\bar M^\star\approx2.5$, fragile $\approx1.2$.
1. Relative-magnitudes sensitivity and the breakdown value¶
createSensitivityResults_relativeMagnitudes takes the event-study betahat and sigma, the number of pre- and post-periods, and a grid of $\bar M$, and returns the robust CI for the first post-treatment effect at each $\bar M$; constructOriginalCS gives the conventional (exact-parallel-trends) CI. The breakdown $\bar M^\star$ is where the robust CI first includes zero. The robust scenario survives to $\bar M\approx2.5$, the fragile one only to $\approx1.2$.
suppressMessages(library(HonestDiD))
load_es<-function(tag){ list(b=as.numeric(read.csv(paste0("es_betahat",tag,".csv"),header=FALSE)[,1]),
S=as.matrix(read.csv(paste0("es_sigma",tag,".csv"),header=FALSE))) }
rob<-load_es(""); fra<-load_es("_fragile")
Mv<-c(0,0.25,0.5,0.75,1,1.25,1.5,1.75,2,2.5) # match the Python grid
rmR<-createSensitivityResults_relativeMagnitudes(betahat=rob$b,sigma=rob$S,numPrePeriods=3,numPostPeriods=5,Mbarvec=Mv)
rmF<-createSensitivityResults_relativeMagnitudes(betahat=fra$b,sigma=fra$S,numPrePeriods=3,numPostPeriods=5,Mbarvec=Mv)
oR<-constructOriginalCS(rob$b,rob$S,3,5); oF<-constructOriginalCS(fra$b,fra$S,3,5)
brk<-function(rm){ h<-rm$Mbar[rm$lb<=0]; if(length(h)) as.numeric(h[1]) else NA_real_ }
cat(sprintf("ROBUST : original CI [%.2f, %.2f]; breakdown Mbar* = %.2f\n", as.numeric(oR$lb),as.numeric(oR$ub), brk(rmR)))
cat(sprintf("FRAGILE: original CI [%.2f, %.2f]; breakdown Mbar* = %.2f\n", as.numeric(oF$lb),as.numeric(oF$ub), brk(rmF)))
options(repr.plot.width=9, repr.plot.height=4.6)
print(createSensitivityPlot_relativeMagnitudes(rmR, originalResults=oR) + ggplot2::ggtitle("Robust scenario: relative-magnitudes sensitivity"))
Warning message: "package 'HonestDiD' was built under R version 4.6.1"
Warning message in .ARP_computeCI(betahat = betahat, sigma = sigma, numPrePeriods = numPrePeriods, : "CI is open at one of the endpoints; CI length may not be accurate"
Warning message in .ARP_computeCI(betahat = betahat, sigma = sigma, numPrePeriods = numPrePeriods, : "CI is open at one of the endpoints; CI length may not be accurate"
Warning message in .ARP_computeCI(betahat = betahat, sigma = sigma, numPrePeriods = numPrePeriods, : "CI is open at one of the endpoints; CI length may not be accurate"
ROBUST : original CI [0.90, 1.29]; breakdown Mbar* = 2.50
FRAGILE: original CI [0.50, 0.89]; breakdown Mbar* = 1.25
2. The fragile scenario, side by side¶
The same plot for the fragile scenario makes the contrast visual: its robust CI crosses zero far sooner. Two event studies with equally modest-looking pre-trends, opposite robustness — the distinction a pre-trends eyeball test cannot make.
options(repr.plot.width=9, repr.plot.height=4.6)
print(createSensitivityPlot_relativeMagnitudes(rmF, originalResults=oF) + ggplot2::ggtitle("Fragile scenario: relative-magnitudes sensitivity (breaks early)"))
3. The smoothness restriction¶
The alternative restriction, $\Delta^{SD}(M)$, bounds how much the differential trend can change slope (its second difference) between consecutive periods by $M$ — formalizing "the pre-trend, if it continued, would extrapolate roughly linearly." createSensitivityResults with method = "C-LF" returns the robust CI as $M$ grows from 0 (exact linear extrapolation) upward. For the robust scenario the effect again tolerates substantial curvature before losing significance.
sdR<-createSensitivityResults(betahat=rob$b,sigma=rob$S,numPrePeriods=3,numPostPeriods=5,method="C-LF",Mvec=seq(0,0.3,by=0.1))
cat("Smoothness (second-difference) sensitivity, robust scenario:\n"); print(as.data.frame(round(sdR[,c("M","lb","ub")],3)))
options(repr.plot.width=9, repr.plot.height=4.4)
print(createSensitivityPlot(sdR, constructOriginalCS(rob$b,rob$S,3,5)) + ggplot2::ggtitle("Robust scenario: smoothness (second-difference) sensitivity"))
Smoothness (second-difference) sensitivity, robust scenario:
M lb ub 1 0.0 0.733 1.407 2 0.1 0.660 1.482 3 0.2 0.572 1.565 4 0.3 0.482 1.659
4. Summary¶
HonestDiD reproduced the sensitivity analysis: under the relative-magnitudes restriction the robust scenario broke down at $\bar M^\star\approx2.5$ and the fragile one at $\approx1.2$ (matching the Python notebook), and the smoothness restriction told a consistent story. The package's plots make the robustness immediately legible.
The workflow to adopt: never report a DiD or event-study estimate without a Rambachan-Roth sensitivity analysis — a breakdown value under relative magnitudes and/or smoothness — and judge it against the confounding trends plausible in the setting. It is the difference-in-differences twin of the matching subsection's Rosenbaum/E-value analysis: both convert an untestable identifying assumption (parallel trends; unconfoundedness) into a quantitative robustness statement. Cross-links: builds directly on the event-study / staggered-adoption estimates of subsection 6; conceptually parallel to the sensitivity-analysis notebook (subsection 2) and the weak-instrument-robust confidence sets (subsection 3). This completes the depth of the Difference-in-Differences subsection.