Weak-Instrument-Robust Inference

Diagnosing Weakness Is Not Surviving It

The instrumental-variables page diagnosed weak instruments with the first-stage $F$ and the $F > 10$ rule of thumb. But diagnosing weakness is not the same as doing valid inference despite it. When instruments are weak the ordinary 2SLS Wald interval, $\hat\beta \pm 1.96\,\widehat{\rm se}$, is simply invalid: the estimate is biased toward OLS, the standard error understates the true uncertainty, and the interval can cover the truth far less than 95% of the time.

A simulation with a known effect of $\beta = 1$ and a confounder biasing OLS upward makes the failure quantitative. The last column previews the alternative developed below: Anderson–Rubin, which tests candidate values of $\beta$ rather than estimating it. Two regimes: a strong single instrument, and thirty weak ones — the structural feature of the Angrist–Krueger design, where quarter of birth was interacted into hundreds of weak instruments.

True effect 1.00. With many weak instruments 2SLS lands most of the way to the confounded OLS value, and the nominal 95% interval covers 4% of the time.

With a strong instrument everything behaves. With many weak ones, 2SLS is dragged from the truth of 1.00 to 1.84, most of the way to the confounded OLS value of 1.99, and the nominal 95% Wald interval covers the truth 4% of the time. That is the Bound–Jaeger–Baker critique made numerical: an interval you must not trust, advertising a precision it does not have. Anderson–Rubin holds its nominal coverage in both regimes.

The Anderson–Rubin Confidence Set

Anderson & Rubin's idea (1949) sidesteps estimating $\beta$ altogether. To test a candidate $\beta_0$, form the residual $Y - \beta_0 D$; if $\beta_0$ is the true effect that residual is exogenous, so the instruments should not predict it. Regress it on the instruments and test that all their coefficients are zero. The AR confidence set is every $\beta_0$ the test does not reject — and because the test is exact at any first-stage strength, the set has correct coverage however weak the instruments are.

On the single weak-instrument sample, the contrast to look for is width, not coverage. The Wald interval comes out at [−1.28, 3.12] while the AR set runs to the edge of the search grid at [−4.00, 6.00], meaning effectively unbounded. Both happen to contain the true value of 1 in this particular draw — coverage is a repeated-sampling property, and no single sample can display it, which is why it was measured over many replications above. What one sample can show is that the Wald interval advertises precision the instruments do not support, while the AR set reports honestly that the data cannot pin the effect down.

Recalibrating “F > 10” to “F > 104.7”

The AR set is fully robust, but for a single instrument many practitioners still want an estimate plus or minus something. Lee, McCrary, Moon & Weidner (2022) provide it. The tF procedure keeps the 2SLS estimate and standard error but replaces the fixed 1.96 with an adjusted critical value $c_F$ that depends on the first-stage $F$. When the first stage is strong $c_F \to 1.96$; when it is weak $c_F$ grows, widening the interval to restore validity.

Their headline recalibrates the folklore, and the size of the correction is the point: to justify the usual $t > 1.96$ at the 5% level, the first-stage $F$ must exceed 104.7 — not 10. The old rule of thumb was more than an order of magnitude too lenient, which is sobering for the many published IV studies whose first stage sat between 10 and 100. Their conventional confidence intervals were too narrow.

Card, Reassessed

Which brings the previous page's result back for reassessment. Card's instrument gives a first-stage $F$ of 13.3 — comfortably past the old rule, and less than an eighth of the way to 104.7.

2SLS estimate 0.1315 with standard error 0.0550; first-stage F = 13.3.

The naive interval's tidy significance does not survive. The Wald interval excludes zero and produces the clean "13% return" headline. Anderson–Rubin, which is both robust and the more powerful of the two corrections, is wider, shifted upward, and just excludes zero. The conservative tF interval, with $c_F \approx 2.93$, includes zero.

So the honest reading is that Card's estimate is imprecise and borderline once weak-instrument uncertainty is taken seriously — real by the more powerful test, not by the conservative one. That is a materially different message from the naive interval, and it is roughly where the returns-to-schooling debate has sat for years. The neatness of the original interval was an artifact of treating $F = 13$ as strong.

One note on provenance, since it changes how the numbers should be read. The critical value $c_F \approx 2.934$ is not derived in the notebook — it is Lee et al.'s critical-value function evaluated at Card's first-stage $F$, as implemented in the ivDiag package and used in the R companion. The AR inversion, the coverage simulation and the 2SLS fit are all computed from scratch.

Where this sits

This is the honest completion of Instrumental Variables: there the first-stage $F$ diagnosed weakness, here inference is made valid despite it. The test-inversion logic — build a confidence set from the values a test cannot reject — is the same idea behind conformal prediction in the machine-learning arc and behind the placebo inference used in synthetic control. And the many-weak-instruments bias is a cousin of the overfitting that cross-fitting guards against in double machine learning — the placebo test here is the same move as the permutation inference in synthetic control and the placebo cutoffs in RD falsification: run the procedure where the answer must be nothing, and see what it returns.

The Instruments That Were Never There

The simulation in the first section stood in for a real study. This is the study. Angrist & Krueger (1991) is among the most cited papers in applied economics and the origin of the many-weak-instruments literature, and its data — 329,509 men born 1930–1939, from the 1980 census — makes the failure concrete rather than stylised.

The design is elegant. Compulsory schooling laws let you leave at 16, but you start school in the year you turn 6, so children born early in the year reach their sixteenth birthday having completed less schooling than those born late. It works: mean schooling runs from 12.688 years for first-quarter births to 12.839 for fourth-quarter. A gap of 0.151 years — six weeks of school — is the entire basis of the design.

Two specifications. The simple one uses three quarter-of-birth dummies. The celebrated one interacts quarter with year and state of birth, giving 180 instruments, and it delivers what it promises: the standard error halves.

329,509 men, controlling for year and state of birth throughout.

Read the last two columns together. Adding 177 instruments bought a 52% reduction in standard error and cost 93% of the first stage, which fell from 36.0 to 2.58. Against this page's own requirement of 104.7, the celebrated specification is short by a factor of 41. The estimate moved too, and in the direction the theory predicts: from the simple specification's 0.1077 back to 0.0928, 37% of the way toward OLS, because a first stage that is mostly noise reproduces the endogenous variation it was brought in to purge.

Then the test that turned the paper into a cautionary tale. Bound, Jaeger & Baker (1995) threw the real quarters away, replaced them with random draws, and ran the identical specification. Here that is done 200 times.

Random numbers, put through the published specification, return an average estimate of 0.0651 — essentially OLS, which is where irrelevant instruments send 2SLS — with a median |t| of 4.4. 197 of 200 draws are significant at 5%, and 8 of 200 produce an estimate larger than the real one. Nothing in the output announces that the instruments are noise.

And the standard errors are not lying. The spread of the placebo estimates is 0.0150 against a mean reported standard error of 0.0150 — calibrated almost exactly. The estimator has an accurate view of its own sampling variability and no view whatsoever of the fact that it is estimating nothing. That is the distinction worth carrying: a standard error answers how much would this move across samples, and never is this quantity identified. No amount of care in computing one will make it answer the second question.

The one honest signal is the first stage. Placebo F never exceeds 1.28 across 200 draws, which is what pure noise gives; the real specification's 2.58 is genuinely more than nothing. Quarter of birth does shift schooling, the exclusion restriction is as defensible as any in the literature, and the design is not fraudulent. It is underpowered by a factor of forty, which no amount of extra instruments repairs — and adding them made it worse while making it look better.

Notebooks

Downloads

References

• Staiger, D. & Stock, J. H. (1997). Instrumental Variables Regression with Weak Instruments. Econometrica 65(3), 557. — weak-instrument asymptotics, and the F > 10 rule
• Stock, J. H. & Yogo, M. (2005). Testing for weak instruments in linear IV regression. Identification and Inference for Econometric Models (eds. Andrews & Stock), 80–108. — critical values for the weak-instrument test
• Andrews, I., Stock, J. H. & Sun, L. (2019). Weak Instruments in Instrumental Variables Regression: Theory and Practice. Annual Review of Economics 11(1), 727–753. — what to do in practice
• Lee, D. S., McCrary, J., Moon, M. J. & Weidner, M. (2022). Valid t-ratio Inference for IV. American Economic Review 112(10), 3260–3290. — the tF correction used here