Vacek (1985) The Effect of Conditional Dependence on the Evaluation of Diagnostic Tests

Summary

Vacek analyzes what happens to the no-gold-standard estimation of diagnostic test error rates when the standard conditional-independence assumption fails. The Hui-Walter (1980) design estimates the sensitivities and specificities of two imperfect tests — plus the prevalences of two populations — by maximum likelihood, assuming the two tests' errors are independent given true disease status. Vacek shows that if the tests are instead conditionally dependent (their errors are correlated within true-status groups), the error rates of both tests are substantially underestimated (the tests look better than they are), and the prevalence estimates are biased in a direction that depends on the two conditional covariances and the prevalence value.

Key Claims

• The setup. Sensitivity $=1-\beta$, specificity $=1-\alpha$; $\alpha,\beta$ are the false-positive and false-negative rates. With a perfect gold standard these are trivial to estimate (Fleiss 1973); with a known-error reference test, Gart-Buck (1966) correct for it.
• Hui-Walter identification. When neither test's error rates are known, applying both tests to individuals from two populations of differing prevalence makes the MLEs of both tests' error rates and both prevalences identifiable — but only under conditional independence.
• Conditional dependence and its parameters. Real tests often share failure modes (same underlying biology, same confounders), so their errors are correlated given true status. Vacek parameterizes this with the conditional covariances of the two tests' results within the diseased and non-diseased groups.
• Bias results. Ignoring conditional dependence (fitting the independence model when covariances are nonzero) leads to (i) underestimated error rates for both tests — i.e. overstated sensitivity and specificity — and (ii) prevalence estimators that can be biased either upward or downward, depending on the relative sizes of the two conditional covariances and the prevalence.
• Implication. No-gold-standard accuracy estimates that assume conditional independence should be treated with caution: correlated test errors make a battery of mediocre-but-similar tests look deceptively accurate.

Concepts Introduced or Extended

• Diagnostic Accuracy

Entities Mentioned

• Pamela M. Vacek

Quotes

"If the tests are conditionally dependent, error rates for both tests can be substantially underestimated. Estimators for the prevalence rates in the two populations can be positively or negatively biased, depending on the relative magnitude of the two conditional covariances and the value of the prevalence parameter."

My Take

This is the paper that put a number on the danger every no-gold-standard diagnostic study lives with: the conditional-independence assumption is not innocuous, and when it fails the failure is optimistic — correlated errors make the tests look more accurate than they are, which is exactly the wrong direction for a screening decision. It sits right beneath the diagnostic-accuracy concept's four-model taxonomy: the CI (conditional-independence) latent class model is the baseline, and Vacek's covariance-augmented view is why the BB / GRE / finite-mixture dependence structures exist at all. Read together with Albert-Dodd (2004) the message compounds — not only can you not tell the dependence models apart with few tests (Albert-Dodd), but choosing the wrong (independence) one biases sensitivity/specificity in a systematic, over-confident direction (Vacek). It is also a clean early instance of the general "local dependence breaks latent class inference" theme that recurs in latent class work.