Benítez-Silva et al 2004 — How Large Is the Bias in Self-Reported Disability

Summary

Using Health and Retirement Study (HRS) waves 1–3 (1992–1996/97) linked to Social Security Administration (SSA) administrative records, Benítez-Silva et al. test whether individuals' self-reported disability status is an unbiased predictor of the SSA's ultimate disability award decision. Five statistical tests — ranging from a simple unconditional mean to nonparametric conditional moment tests — fail to reject the Rational Unbiased Reporting (RUR) hypothesis against SSA's ultimate award: individuals who self-report a health limitation preventing work receive ultimate Disability Insurance (DI) awards at the same rate as individuals the SSA would independently classify as disabled. The same tests decisively reject unbiasedness for the SSA's first-stage Disability Determination Services (DDS) decision, indicating that the appeal-and-review process, not the initial screening, is the stage that delivers unbiasedness. Prior studies (especially Kreider 1999) overstated self-report bias by imposing strong parametric assumptions; the present nonparametric approach eliminates that artifact.

Key Claims

• RUR cannot be rejected for ultimate award. Five tests (unconditional $p = 0.33$, moment restrictions $p = 0.19$, ordinary least squares (OLS) $p = 0.11$, Bierens 1990 $p = 0.09$, Horowitz-Spokoiny 2001 $p = 0.79$) all fail to reject $E[\tilde{a} - \tilde{d} \mid x] = 0$ at conventional levels. $n = 393$ (unconditional), $n = 356$ (conditional on covariates).
• RUR is rejected for first-stage DDS decisions. All five tests reject ($p = 0.00$) for the initial DDS determination: SSA initial screeners are systematically harsher than self-assessors. The appeal process functions as a self-selection correction mechanism that restores unbiasedness at the ultimate-award stage.
• hlimpw is an approximate sufficient statistic. The HRS variable "Does a health problem keep you from working altogether?" approximately captures what the SSA evaluates in the full determination process. About $20\%$ of DI recipients self-report no work limitation; $5\%$ report earnings above substantial gainful activity (SGA) — consistent with survey noise, not systematic over-reporting.
• Bivariate probit (one-type model). Likelihood-ratio (LR) test statistic $38.4$ fails to reject $\beta_a = \beta_d$ at the $5\%$ level; $\rho = 0.206$ (moderate correlation between SSA and individual unobservables).
• Two-type model. LR statistic one-type vs. two-type $= 75.66$ (strongly rejects one-type); restricted two-type LR $= 68.11$ ($p = 0.067$); roughly $60\%$ Type I (straightforward cases) and $40\%$ Type II (ambiguous cases).
• LFP collapse at disability onset. Figure 1: labor force participation (LFP) drops from $\approx 60\%$ to $\approx 15\%$ in the month of onset, further kinks at application, and stabilizes near $\approx 5\%$ after award. This pattern validates that hlimpw onset dates mark genuine work-limiting events.
• Kreider (1999) comparison. The large self-report biases found by Kreider are entirely an artifact of imposing parametric functional form assumptions. Nonparametric tests on the same underlying question produce no evidence of bias.

Concepts Introduced or Extended

• Rational Unbiased Reporting
• DI Classification Errors
• SSA Sequential Determination Process

Entities Mentioned

• Hugo Benítez-Silva
• Moshe Buchinsky
• Hiu Man Chan
• Sofia Cheidvasser
• John Rust
• Health and Retirement Study

Quotes

"Our test results show that we cannot reject the Rational Unbiased Reporting (RUR) hypothesis … even though we use a rich set of covariates and both parametric and nonparametric test methods."

"These results suggest that self-reported health as a proxy for true disability is a better measure than is commonly believed."

My Take

The central methodological lesson is that parametric restrictions are doing the heavy lifting in studies that find large self-report bias. Once you move to nonparametric conditional moment tests, the apparent bias evaporates. The paper's limitation — acknowledged — is that the ultimate SSA award is itself potentially endogenous (it depends partly on the applicant's self-report). The paper does not directly resolve whether individuals are accurately assessing objective functional capacity, only that their assessments align with SSA's final determinations. The two-type extension is interesting but the interpretation of Type I vs. Type II is not fully pinned down.