Moffitt 2003 — Causal Analysis in Population Research

Summary

An 11-page methodological essay in Population and Development Review (2003, 29(3): 448–458) written for a demography audience. Moffitt explains the economist's counterfactual framework for causal inference, the problem of endogeneity, the role of instrumental variables (IV) as the standard solution, and the fundamental tradeoff between internal validity (credible exogeneity) and external validity (generalizability). An extended applied example on teenage childbearing illustrates how each identification strategy works in practice and where it falls short. The conclusion calls for synthetic, evidence-weighing approaches rather than privileging any single identification strategy.

Key Claims

• Every causal question rests on an unobservable counterfactual: the outcome individual i would have had under the treatment they did not receive. Causal effects cannot be estimated without minimal untestable assumptions; there is no assumption-free causal analysis.
• When the variable of interest is endogenous (correlated with unobserved outcome determinants), ordinary least squares (OLS) is invalid. The standard remedy is an exclusion restriction: an instrument that shifts the endogenous variable but does not directly affect outcomes or correlate with their unobservables.
• Any IV estimate identifies only the average effect for the subset of the population whose treatment status is actually changed by the instrument — the "switchers" or local compliers (local average treatment effect, LATE: Imbens and Angrist 1994). A price change moving smoking rates from 30% to 40% identifies only the 10% who switched, not the full population.
• Four instrument types: (1) ecological/area variables (unemployment rate, policy laws); (2) demographic difference-in-differences (DiD) models (comparing group trends); (3) twin/sibling models (within-family variation); (4) natural experiments (narrow, sudden, quasi-random variation).
• Natural experiments maximize internal validity at the cost of external validity: they identify effects for very narrow subpopulations (4–6-year-olds in two states; women who miscarried) that may not generalize.
• Maximizing internal validity alone produces "a collection of miscellaneous facts that do not add up to any general knowledge." The solution is to weigh evidence from studies with different mixes of internal and external validity, combine formal theory with informal evidence, and focus on convergence across identification strategies.
• "Modesty of claims for truth is the clear lesson."

Concepts Introduced or Extended

• Instrumental Variables
• China Trade Shock (Bartik IV as one application of ecological/area instruments)
• Regression Kink Design

Entities Mentioned

• Robert Moffitt

Quotes

"There is no mechanical algorithm for producing a set of 'assumption free' facts or causal estimates based on those facts." (citing Heckman 2000)

"A collection of miscellaneous facts that do not add up to any general knowledge... is intellectually constricting and does not allow the pursuit of the larger social science questions and the explanation of societal-wide trends that motivate much of social science research."

"Modesty of claims for truth is the clear lesson from this review."

My Take

A lucid, balanced methodological essay. Moffitt is more skeptical of the natural experiment movement than most practitioners — he explicitly warns against the trade of external for internal validity, which was a contrarian position in 2003 and remains important. The LATE framing is essential for interpreting every IV paper in this wiki: examiner-lottery IV (Maestas, Mullen, Strand 2013; French and Song 2014) identifies effects for the complier population of marginal applicants, not the full enrolled or rejected population. The essay's strength is that it names this limitation clearly and argues for evidence synthesis rather than method fetishism. Limitation: the applied example (teenage childbearing) is dated and not relevant to this wiki's focus; the essay is most useful as a conceptual reference.