Meseguer 2013 — Outcome Variation in the Social Security Disability Insurance Program The Role of Primary Diagnoses

Summary

Using a Bayesian hierarchical multinomial logit model on a 10% sample of the Disability Research File (462,578 applications, 1997–2004), Meseguer classifies each application into one of four mutually exclusive adjudicative outcomes and decomposes outcome variation into individual-level, state-level, and diagnosis-level components. The central finding is that primary diagnosis accounts for approximately 44% of total variation in initial allowance decisions — far more than state of residence (~6%) or any combination of individual-level covariates. A diagnosis-only model outpredicts a full model including age, sex, earnings, employment status, and state. High positive correlation (r ≈ 0.56–0.74) between diagnosis-level initial and final allowance predictions indicates the ordinal ranking of impairment severity is preserved across adjudicative stages.

Key Claims

• Diagnoses dominate variance: Intraclass correlation coefficient (ICC) for initial allowances by diagnosis = 43.89% (standard deviation [SD] 2.70); by state = 6.22% (SD 1.17). Knowing the primary diagnosis explains more variation in outcomes than all demographic and labor-market variables combined.
• Outcome distribution (1997–2004): Initial allowances 46.2%; initial denials not appealed 19.4%; final allowances 24.9%; final denials 9.5%. Combined ~71% ultimately allowed, ~29% ultimately denied.
• Diagnosis-level extremes: Malignant neoplasm of lung ~94% initial allowance rate; degenerative/discogenic back disorders ~23%. The spread across 181 diagnosis codes is enormous.
• Ranking preserved initial to final: Correlation between diagnosis-level initial and final allowance predictions = 0.737 (intercepts-only) / 0.561 (with individual controls). Diagnoses with higher initial allowance propensity also have higher final allowance propensity — the system is internally consistent, not arbitrarily variable at the hearing stage.
• State-level variance is modest: Most observed state-to-state variation in allowance rates reflects the diagnostic mix of applicants within each state, not inconsistent adjudication across Disability Determination Services (DDS) agencies. The state-level initial/final allowance correlation is near zero (0.048) — unlike the diagnosis-level correlation, there is no stable state-level ranking.
• Individual-level predictors: Males and higher earners have higher initial allowance probability; unemployed applicants and reapplicants have lower initial allowance — consistent with the Conditional DI Applicants mechanism (marginal applicants with labor-market attachment are less impaired on average). Age effect on initial allowance is U-shaped: elevated at very young ages (mental impairments) and at older ages (musculoskeletal/cardiovascular).
• Model fit: Diagnosis model deviance information criterion (DIC) = 980,212 (best); state model DIC = 1,080,995; pooled model DIC = 1,093,989. Correct prediction rates: 55.3% (diagnosis model with predictors) vs. 48.5% (state model with predictors) vs. 47.5% (pooled).

Concepts Introduced or Extended

• SSA Sequential Determination Process
• Listing of Impairments
• Conditional DI Applicants
• Disability Insurance Program

Entities Mentioned

• Javier Meseguer
• Social Security Administration
• Alexander Strand
• Bernard Wixon

Quotes

"Medical diagnoses play a substantial role in explaining individual-level variation in initial allowances... the ordinal ranking of impairments between these two adjudicative outcomes is widely preserved."

"Primary diagnosis codes carry greater predictive ability than all other variables combined. Knowing impairments yields more accurate classification than knowing age, sex, state, earnings, employment status, and application history."

My Take

This is the most rigorous quantification of what practitioners have always known: the diagnosis is the decisive variable. The 44% ICC is striking — it means that if you randomly pick two disability insurance (DI) applicants from the same state with similar age, sex, and earnings, but different primary diagnoses, the diagnosis explains nearly half the difference in their outcomes. The state-level ICC of 6% does not mean state variation is trivial in absolute terms, but it does suggest that most apparent state variation reflects diagnostic composition rather than inconsistent adjudication. The finding that state initial/final correlation is near zero while the diagnosis initial/final correlation is 0.56–0.74 is the clearest evidence that diagnosis-level variation is systematic (reflects genuine severity differences) while state-level variation is idiosyncratic noise. The paper's methodological contribution — applying Bayesian multilevel multinomial logits to DI adjudicative data — is also notable for operationalizing the four-outcome classification that is often collapsed to a binary in simpler analyses.