Principal Stratification and Weak Instruments

Compliance Types as Latent Classes

The noncompliance notebook recovered the complier effect in the Sommer–Zeger vitamin A trial with Bloom's estimator, and confirmed it is numerically identical to two-stage least squares. The quantity is the complier average causal effect (CACE) — the effect among children who would take the vitamin if offered — and it is recovered by dividing one intention-to-treat contrast by another: ${\rm ITT}_Y$, the effect of being assigned the vitamin on mortality, over ${\rm ITT}_D$, the effect of being assigned it on actually taking it. That is correct, and it is a ratio — one estimate divided by another, which is a more fragile object than a difference. When the denominator is large, as it is here at ${\rm ITT}_D = 0.80$, nobody notices. The question is what happens when it is not, and the answer turns out to be less lurid and more useful than the standard warning suggests.

The Bayesian route approaches the same problem from the other end. Compliance type is a latent class: each child is a complier or a never-taker, and which one is never observed. The trial has one-sided noncompliance — children assigned vitamin A could decline it, controls had no access — so the treated arm separates cleanly into compliers who took it and never-takers who refused, while the control arm is a mixture of the two, indistinguishable. That mixture is the entire difficulty, and principal stratification (Frangakis & Rubin 2002; Imbens & Rubin 1997) writes it down explicitly rather than dividing it away.

Writing $\pi_c$ for the complier share, $\mu_{c1}, \mu_{c0}$ for complier mortality under treatment and control, and $\mu_n$ for never-taker mortality, the three observed cells are pure, pure and mixed respectively. The complier effect is $\mu_{c1}-\mu_{c0}$, and the exclusion restriction is what permits never-taker mortality to be shared across arms. The trial enters as its published cell counts — 9,675 took it with 12 deaths, 2,419 refused with 34 deaths, 11,588 controls with 74 deaths — the same table the frequentist notebook expands into individual records.

On the real trial the two agree, which is the check rather than the finding: the posterior gives −3.15 with 95% interval [−5.44, −0.86] against Bloom's −3.23, and $\Pr(\text{CACE} < 0) = 0.997$. A strong instrument makes the ratio well behaved, and any correct model has to reproduce it.

What the model adds is structural. Never-taker mortality comes out near 14 per 1,000 against complier-under-control mortality of 4.5 — never-takers were dying at 3.2 times the rate compliers would have died at untreated. That is precisely the selection which makes as-treated and per-protocol comparisons biased, and here it is a parameter with a posterior rather than something read off the difference between two published rates.

What Actually Degrades When the Instrument Weakens

The real trial has an unusually cooperative instrument. Most encouragement designs do not: a mailing, a reminder or a small incentive might move take-up by ten or twenty points rather than eighty. So the notebook simulates trials with a known complier effect at compliance rates from 0.80 down to 0.05 and grades both approaches on the two things that matter — does the interval cover the truth, and how wide is it. Both are 95% intervals computed on the same simulated trials, so the comparison is like for like.

Bloom's estimator does not break. Its coverage is at nominal across the whole range, and its median estimate never leaves the truth. Nothing about it becomes invalid. What fails is informativeness: the interval widens from 7.98 to 165.50 around an effect of size 3, and at the weakest compliance rate nearly two-thirds of trials return an estimate exceeding 20 per 1,000 in magnitude.

True effect −3.0 per 1,000; 2,000 simulated trials per row, 6,000 children per arm.

The last column explains why the textbook pathology never arrives. That story — a ratio whose sampling distribution turns heavy-tailed because the denominator might be near zero — requires the denominator to be uncertain relative to its own size. At 6,000 children per arm the coefficient of variation of $\widehat{\rm ITT}_D$ tops out at 5.6%, so it never approaches zero. The damage is pure $1/{\rm ITT}_D$ amplification of noise that was already in the numerator, which is enough to wreck the estimator without any distributional drama.

The Bayesian version does not rescue it, and how it fails is the interesting part. It over-covers — 100% where 95% was asked for, at four of six compliance rates — while returning intervals roughly a third the width of Bloom's. Over-coverage is a calibration failure too, in the safe direction: the intervals are conservative rather than sharper. All 600 fits converged, none exceeding an r-hat of 1.01 — the standard check that independent sampler chains have settled on the same distribution, where anything above about 1.01 says they have not.

Bayesian column from 100 simulated trials per row (Monte Carlo error on coverage at most ±2.4 points), Bloom from 2,000.

Where the Narrowness Comes From

Narrower and covering more means information arrived from somewhere other than the data, and the obvious suspect is the prior: ${\rm Beta}(1,50)$ on each mortality rate rules out values the ratio estimator happily entertains. That is testable rather than assertable, so the weakest instrument is refitted under three priors of increasing vagueness.

All at ITT_D = 0.05, the weakest instrument in the sweep; true effect −3.0 per 1,000.

The prior is confirmed as part of the answer: widening it widens the interval monotonically and walks coverage back down toward nominal, so the over-coverage above is substantially a prior artefact.

It is not the whole answer, though. Even ${\rm Beta}(1,5)$ — prior mean mortality of 167 per 1,000, vaguer than anyone would defend for children — leaves the interval at 97.9 against the ratio's 165.5. Widening the prior as far as is remotely reasonable closes only about a third of the gap. The remainder is the model structure: writing the control arm as an explicit mixture and imposing the exclusion restriction extracts something division does not, and that part survives however vague the prior is made.

The general lesson is that an estimator can be entirely valid and entirely useless at the same time, and coverage alone will not distinguish the two. Bloom's interval passes every calibration check while being far too wide to support a decision; the Bayesian interval is narrower, and part of that narrowness is borrowed from the prior rather than earned from the data. Which failure is preferable is a question about how much you are willing to assume, which is the question this group is arranged around.

Where this sits

The frequentist treatment establishes CACE and its identity with 2SLS, assumed rather than re-derived here. The latent-class structure is the machinery of the latent-class arc doing causal work, applied to compliance rather than to survey responses. And the weak-instrument problem returns in earnest whenever an instrument is found rather than assigned — it appears here in its mildest possible form, on a randomized encouragement with thousands of children per arm.

Notebook

Data

No download accompanies this page. The vitamin A trial enters as its six published cell counts — 9,675 took it with 12 deaths, 2,419 refused with 34 deaths, 11,588 controls with 74 deaths — which are written out in the notebook's first cell. Everything in the weak-instrument sections is simulated from seeded generators, so the whole page reproduces from the notebook alone.

References

• Frangakis, C. E. & Rubin, D. B. (2002). Principal Stratification in Causal Inference. Biometrics 58(1), 21–29. — principal stratification
• Angrist, J. D., Imbens, G. W. & Rubin, D. B. (1996). Identification of Causal Effects Using Instrumental Variables: Comment. Journal of the American Statistical Association 91(434), 465. — the noncompliance case it generalises
• Bloom, H. S. (1984). Accounting for No-Shows in Experimental Evaluation Designs. Evaluation Review 8(2), 225–246. — the simplest version of the same adjustment