Burkhauser Houtenville and Tennant 2014 — Capturing the Elusive Working-Age Population With Disabilities Reconciling Conflicting Social Success Estimates

Summary

The paper demonstrates that the 2008 replacement of the Current Population Survey Annual Social and Economic Supplement (CPS-ASEC) work-activity limitation question with the American Community Survey (ACS) / CPS Basic Monthly Survey (CPS-BMS) six-question disability sequence creates systematically biased estimates of employment and Social Security Disability Insurance (SSDI) / Supplemental Security Income (SSI) receipt among working-age people with disabilities. Using 2010 CPS data, the authors show that the two measures identify only partially overlapping populations and that the six-question-only population substantially overstates employment ($30.8\%$ vs. $16.6\%$ for the work-limitation-only group) and understates program participation ($38.9\%$ vs. $49.2\%$). A face validity test against known SSDI/SSI recipients shows the six-question sequence captures only $63.3\%$ of beneficiaries versus $84.1\%$ for the work-limitation question; the combined seven-question sequence captures $92\%$.

Key Claims

• The work-activity limitation population (B+C) and the six-question sequence population (A+B) are not concentric — they are only partially overlapping. In the 2010 CPS: A (six-q only) $= 3.3\%$, B (both) $= 4.6\%$, C (work-limit only) $= 3.7\%$; combined A+B+C $= 11.6\%$ vs. $7.9\%$ or $8.3\%$ for either alone.
• Using the six-question sequence alone overstates employment ($30.8\%$ vs. combined $28.2\%$) because six-q-only population (A) has $57.3\%$ employment — far above the work-limit-only population (C) at $22.8\%$.
• Using the six-question sequence alone understates SSDI/SSI-Disabled Adults receipt: $38.9\%$ (A+B) vs. $49.2\%$ (B+C) vs. $37.6\%$ for C-only vs. $11.6\%$ for A-only.
• Face validity test: of $7{,}337{,}059$ SSDI/SSI-Disabled Adults recipients in 2010 CPS, $63.3\%$ are captured by six-question sequence, $84.1\%$ by work-activity limitation, $92.0\%$ by the seven-question union — work-activity limitation's marginal contribution ($28.7$ percentage points (pp)) dwarfs the six-question sequence's marginal contribution ($7.9$ pp).
• Despite large level differences, employment and program participation trends are similar across all disability definitions (1981–2010 CPS; 2005–2010 ACS).
• Disability prevalence is stable at $\approx 8\%$ using work-activity limitation (1981–2010, no trend); the seven-question combined sequence gives $\approx 11.6\%$.
• The ACS 2007 face validity results differ from CPS 2009 due to question placement: in the ACS, work-activity limitation was the last question in the sequence, potentially subject to fatigue/anchoring effects not present in the CPS where the questions are in separate survey modules.
• International Classification of Functioning, Disability and Health (ICF) conceptual model (World Health Organization (WHO) 2001): disability = health condition-based impairment, activity limitation, and/or participation restriction. The seven-question combined sequence is the best practical operationalization; neither alone fully achieves this.
• Recommendation: reinstate the work-activity limitation question in ACS and CPS-BMS, as urged by the American Statistical Association President (Keller-McNulty 2006) but not adopted.

Concepts Introduced or Extended

• Disability Prevalence Measurement
• Disability Insurance Program

Entities Mentioned

• Richard V. Burkhauser
• Andrew J. Houtenville
• Jennifer Tennant
• David Stapleton

Quotes

"In 2010, the CPS six-question sequence missed one third of SSDI/SSI-Disabled Adults recipients, people who should be included in any ICF-based conceptualization of the population with disabilities."

"We find that despite its greater number of disability questions, the six-question sequence alone is less able to identify SSDI and SSI-Disabled Adults beneficiaries than the work-activity limitation question alone."

My Take

The core methodological point is compelling and the face validity test is decisive: if the six-question sequence misses $37\%$ of SSDI/SSI recipients — people with medically-verified work-preventing impairments who unambiguously belong in any ICF-based disability population — then the sequence has a systematic blind spot. The similar-trends finding is important for historical research using CPS work-activity limitation data (trends are valid, levels are not), but does nothing to rescue the six-question sequence as a cross-sectional policy tool. The policy stakes are high: Bureau of Labor Statistics (BLS) disability statistics, employment ratios, and SSDI/SSI take-up rates reported since 2008 are all based on the biased six-question population.