Missing Data
Almost every dataset has holes in it, and almost every analysis quietly deletes the rows containing them. That default is defensible under exactly one condition — that the data are missing completely at random — and it is rarely checked. The Bayesian treatment is disarmingly direct: a missing value is just another unknown, so give it a distribution and sample it alongside the parameters. That is data augmentation, and it is the same move this collection already uses for the latent variable in a probit model, the censored draw in a Tobit, and the class label in latent class analysis. Missing data is the general case of which those are instances.
The arc runs along the line that matters most, which is not technique but what you are willing to assume. Under MCAR and MAR the missingness mechanism is ignorable and imputation needs no model of why data went missing. Under MNAR — when the probability of being missing depends on the value that is missing — no ignorable method can help, and the observed data are formally silent on the question. The last three projects confront that directly: selection models and pattern-mixture models are the two ways to factorise the same joint distribution, they cannot be tested against each other from the data alone, and the honest output is a sensitivity analysis rather than an estimate — δ for a continuous endpoint, ψ for a categorical one. The arc closes by leaving continuous data behind altogether, on partially classified contingency tables where the same augmentation turns out to be the operation a latent class model performs on a missing item. A recurring theme throughout is being precise about how strong the evidence actually is — an imputed mean that shifts in the predicted direction by less than its own credible interval is suggestive, not decisive, and saying so is part of doing the analysis properly.
Which mechanism am I in, and can I find out? The three-way taxonomy only helps if you can place your data in it, and the honest answer is that you can place it halfway. MCAR is testable: if values go missing for reasons unrelated to anything, then the rows with a gap and the rows without should look alike in every observed variable, and that is a comparison you can actually run. It is also almost always rejected. The second step is the one that has no test. Distinguishing MAR from MNAR means asking whether missingness depends on the values you did not get to see — and by construction the data contain nothing about them. Two datasets identical in every observed respect can have come from either mechanism, so no diagnostic, however clever, can separate them. This is not a gap waiting for a better method; it is a fact about what the data are. Everything the last three projects below do follows from it: if the assumption cannot be tested, the only responsible move is to state it explicitly and show how much the conclusion depends on it.
Why once is not enough, and what coverage means. Filling a gap with a single plausible value produces a dataset that looks complete and is treated as complete — so every standard error downstream is computed as though the filled-in numbers were observed. The damage is measurable, and the measurement is coverage: generate data where the truth is known, build a nominal 95% interval, and count how often it actually contains the truth. A calibrated procedure hits 95%. Single imputation reaches 77% — roughly one interval in four misses when only one in twenty should. Imputing several times fixes it because the imputations disagree, and that disagreement is itself the missing information. Rubin’s rules combine them by adding two sources of variance: the average uncertainty within each completed dataset, plus the variance between them. Single imputation omits the second term entirely, which is precisely the 18 points of coverage it loses.
The imputation model is part of the analysis. It is tempting to treat filling the holes and analysing the result as separate jobs, with the first a tidy-up before the real work. They are not separable. The imputations carry only the relationships the imputation model was told about, so any structure it omits is erased from the completed data — and the later analysis, finding no such structure, faithfully reports its absence. The requirement that the imputation model be at least as rich as the analysis model is called congeniality, and two of the results below are the same violation in different clothes. Imputing missing predictors without using the outcome flattens a true coefficient of 1.50 to 0.84, because imputations that ignore y carry no information about the relationship with y. And mice’s default imputer, which copies values from similar donors and so can never go outside the observed range, moves a coefficient from 49.2 to 34.4 on 25 rows — further than the pooling rule it is usually credited to. The rule of thumb that follows is uncomfortable but simple: when in doubt, put more into the imputation model than the analysis needs, including the outcome.
What deletion costs, what one imputation hides, and what cannot be tested
All three panels come from the runs the notebooks below publish. A is the foundations simulation at 45% missingness, B the repeated-sampling coverage experiment from the MICE project, and C the pattern-mixture sensitivity sweep.
A is the argument against the default. The same 6,000 rows lose the same half three times, and only the mechanism differs. Under MCAR the survivors are a smaller copy of the original — the two curves sit on top of each other and the observed mean is +0.01, so deleting is wasteful but honest. Under MAR the surviving distribution has visibly slid left and the mean lands at −0.44; under MNAR it slides further, to −0.58. Nothing about the deletion changed — the same proportion of rows went in each case. What changed is which rows, and that is enough to move an estimate half a standard deviation away from a truth of zero.
B is why one filled-in value is not enough. Filling each gap once and analysing the result as though it were complete produces intervals that contain the truth 77% of the time when they claim 95% — roughly one miss in four where one in twenty was advertised. The imputations disagree with each other, and that disagreement is information about how much the missingness cost; single imputation throws it away, and Rubin's rules put it back as the between-imputation variance.
C is what honesty looks like once MAR is no longer defensible. Sicker treated patients drop out, so complete-case reports a drug effect of −1.79 against a truth of −1.19 — and imputing under MAR barely helps (−1.70), because it fills the dropouts in to resemble the completers, which is the very assumption that caused the bias. Since δ cannot be estimated, the output is the whole curve rather than a point: the benefit stays significant until the treated dropouts would have been 4.4 points worse than completers, which is 2.5 standard deviations of the endpoint and 3.7× the effect being measured. That is the honest report — not "the drug works", but "the drug works unless the dropouts were much worse than anything else in the trial suggests". A section of this arc exists because that sentence cannot be shortened.
How the six examples relate
Ordered by the strength of what you have to assume. The first three need only that the missingness is ignorable; the last three are what to do when it is not.
Ignorable — MCAR or MAR
The mechanism need not be modelled; the data model does the work.
1 · Foundationsdata augmentation under a multivariate normal, and what deletion costs 2 · Chained equationsone regression per variable, so columns can be binary; Rubin's rules for pooling 3 · Missing covariatesholes in the predictors — impute conditional on the outcome, or lose the slopeNot ignorable — MNAR
Two ways to factorise the same joint distribution; the data cannot choose between them.
4 · Selection modelsp(y)·p(r|y) — Heckman's correlated errors, where ρ is the non-ignorability 5 · Pattern-mixture modelsp(y|r)·p(r) — the assumption becomes one visible knob, δLeaving continuous data
A skipped question is not missing — it narrows the answer to a set of cells.
6 · Categorical nonresponseDirichlet–multinomial augmentation, and ψ as the categorical δThe augmentation step is the same throughout, and the same one used for the latent variable of a probit, the censored draw of a Tobit and the class label of a latent class model. Part 6 closes the circle by showing that treating a skipped item as partially classified is what latent class analysis already does.
Foundations — Multivariate Normal
The engine and the ground rules. A missing value gets a distribution and is sampled with the parameters — data augmentation, two conjugate steps under a multivariate normal: impute each row's gaps from their conditional normal given what was observed, then draw μ and Σ from the completed data. What licenses it is Rubin's taxonomy, and the cost of ignoring it is measured: under MAR with 45% missingness, complete-case deletion estimates −0.30 against a truth of 0.00 while discarding 1,838 of 4,000 rows, and both imputation routes land on −0.02. Equally important is where the method stops — under MNAR the sampler is still badly biased (−0.37), because the observed data are silent about the values that are missing. That failure is what the last three projects in the arc exist to address. On the real airquality data the evidence is worth reading at its strength: the imputed days average 40.5 ppb against 42.1 observed, in the direction MAR predicts but by 1.6 ppb with an interval of [33.0, 50.9] — suggestive rather than decisive.
View example →Multiple Imputation by Chained Equations
One joint model is elegant until a column is binary. Chained equations impute
each variable from its own regression on the others — linear for continuous, probit for binary
— cycling until the fills settle, so the imputations behave like draws from a joint
distribution nobody had to write down. The case for doing it multiple times rests on
coverage: single imputation contains the truth in a nominal 95% interval only
77% of the time, and Rubin's rules restore 95% by adding back the
between-imputation variance. The degrees of freedom are where small samples bite:
Rubin's 1987 formula assumes the complete-data df is effectively infinite, and
on nhanes it returns
213–273 df from 25 observations whose complete-data df is 21. That is
impossible on its face; the Barnard–Rubin correction gives 9.6–10.7, which is the difference
between an honest t interval and one far too narrow. The engines also part company on the
coefficients themselves — age 34.4 against 49.2 — because R's mice defaults to
predictive mean matching, which copies a value from a similar donor and so cannot impute
outside the observed range. On 25 rows the choice of imputer moves the answer further than the
pooling rule does.
Missing Covariates in a Regression
The commonest case in practice: the outcome is observed, the predictors have holes. The joint model writes a distribution for the covariates and augments the missing ones inside the regression — imputing each one conditional on the observed predictors and the outcome. That last clause is the entire point, and the module exposes a switch to turn it off so the failure can be produced rather than described. Against a true coefficient of 1.5 the joint model recovers 1.50 while imputing from the other predictors alone collapses it to 0.84; on the real data R's mice shows the same attenuation, 7.65 falling to 5.39. The diagnostic makes the mechanism visible — outcome-blind imputations are flat across y, erasing exactly the information the slope depends on. One result here runs against the usual advice: for missing covariates complete-case deletion is unbiased whenever missingness does not depend on y given x, a weaker condition than MCAR — so here deletion is defensible, just wasteful, discarding 42 of 153 days.
View example →Selection Models — Missing Not at Random
The line the first three projects did not cross. When a value is missing because of what it would have been, no imputation from the data model helps, and the remedy is to model the missingness itself: Heckman's two equations with correlated errors, where the correlation ρ is the non-ignorability. At ρ = 0 it collapses to OLS exactly; at ρ = 0.7 OLS returns an intercept of 1.36 against a truth of 1.00 while the sampler recovers 0.94. But ρ is identified partly by functional form, so an exclusion restriction is close to essential — removing it widens the posterior 2.5-fold. The real data land the point harder than the simulation: on Mroz, ρ = +0.00 with an interval of [−0.27, +0.25], so the honest verdict is not "there is no selection" but "the data cannot tell". The engines then diverge instructively — R's maximum likelihood shifts the returns to education 22% on the strength of a ρ it cannot pin down (0.03, SE 0.15), because the inverse-Mills term is nearly collinear with education, while the Bayesian posterior declines to move. Same weak identification, two symptoms.
View example →Pattern-Mixture Models — MNAR by Sensitivity Analysis
The other MNAR factorisation. Where a selection model writes p(y)p(r|y), pattern-mixture writes p(y|r)p(r) — stratify by missingness pattern and mix. The two describe the same joint distribution and the data cannot choose between them, so the question is which assumption you would rather state out loud. Pattern-mixture makes it impossible to hide: the outcome among the missing is never observed, so it becomes a single interpretable knob δ, and the output is a sensitivity analysis rather than an estimate. A simulated trial shows why that matters — with sicker treated patients dropping out (30% against 3%), complete-case reports −1.79 against a truth of −1.19, and MAR imputation does not rescue it (−1.70), because imputing dropouts to resemble completers repeats the very assumption that caused the bias. Sweeping δ gives the tipping point: 4.4 here, which is 2.5 standard deviations of the endpoint and 3.7× the effect being estimated — so the conclusion is fairly robust. On the real antidepressant trial (28 patients, dropout 5 against 1) the verdict is humbler: promising but unproven, with the interval already spanning zero at δ = 0.
View example →Categorical Missing Data & Nonresponse
The arc closes by leaving continuous outcomes behind. A respondent who answers one question and skips another is not missing but partially classified — we know which cells of the table they could be in — and the tool is the categorical twin of the Gaussian augmentation that opened the arc: a Dirichlet–multinomial Gibbs sampler. On Congdon's 315-respondent survey the natural claim, that folding in the 91 partial answers buys a better estimate, turns out to be half right. The association between the two questions is settled entirely by the 224 who answered both: no partial unit answered both, so the odds-ratio posterior does not move (7.72 against 7.77). What the partial answers do buy is the marginal of the question they did answer — P(Y₁ = yes) shifts 0.544 → 0.571 and narrows 15% on 75 extra answers, while P(Y₂ = yes), with 2 extra, is unchanged. A ψ sweep handles nonignorable skipping as the pattern-mixture δ did. And the augmentation's allocation step is how latent class analysis treats a missing item — the point where the two arcs meet.
View example →