Summary
Keane (2010) defends structural econometrics against the "experimentalist" position advanced by Angrist-Pischke (2010) in the same Journal of Economic Perspectives issue. The paper's central claim is that the structural–experimental dichotomy is false: every identification strategy — instrumental variables (IV), regression discontinuity, difference-in-differences (DID), randomized experiments — requires untestable prior assumptions. Structural models make those assumptions explicit; reduced-form methods embed equally strong assumptions (monotonicity for the local average treatment effect, LATE, parallel trends for DID, exclusion restriction for IV) that are merely invisible rather than absent. Keane argues the correct criterion for evaluating either approach is external holdout validation, not a priori plausibility of assumptions, and points to marketing as the discipline where structural methods have produced the most durable empirical consensus.
Key Claims
- The structural–experimentalist dichotomy is false: all econometric inference requires unverifiable prior restrictions; the difference lies in transparency, not in the strength of assumptions imposed.
- Instrumental variables estimates of LATE require monotonicity (no defiers), exclusion, and independence of the instrument — restrictions as demanding as typical structural assumptions, but less visible.
- Leamer (1983) "Let's Take the Con out of Econometrics" correctly diagnosed specification searching as the central pathology in structural econometrics; the response should have been more disciplined structural modeling, not retreat to reduced form.
- Marketing provides the strongest counter-example: Erdem-Keane (1996) dynamic discrete-choice brand choice model was validated against lab experiments and new data — a consensus on switching costs and price elasticities emerged that is rare outside experimental psychology.
- Keane-Moffitt (1998) structural model of multiple welfare program participation, estimated without randomization, closely predicted actual enrollment patterns after the 1996 U.S. welfare reform (Personal Responsibility and Work Opportunity Reconciliation Act, PRWORA) — external validation without a holdout experiment.
- Structural models support counterfactual policy simulations beyond the observed data range (new prices, new policies); reduced-form methods typically cannot extrapolate beyond the identifying variation.
- Good data — exogenous variation, long panels, large cross-sections — improves both structural and reduced-form estimation; the approaches are not in competition on data quality.
- "Good instruments" are rare in observational data; structural methods can exploit variation from multiple weak sources simultaneously via parametric restrictions, whereas reduced-form IV requires a single strong excluded instrument.
- External holdout validation — predicting outcomes in a new time period, regime, or policy environment not used in estimation — is the right criterion for evaluating models of either type.
- Keane-Wolpin (1997) career decisions model achieved external validation: estimated on National Longitudinal Survey of Youth (NLSY) respondents through age 26, it accurately predicted occupational choices and wages at ages 27–40 without refitting.
Concepts Introduced or Extended
Entities Mentioned
Quotes
"All empirical work in economics involves assumptions that cannot be tested with the data at hand. The assumptions in structural models are just more explicit."
My Take
A pointed and largely correct methodological intervention. The core argument — all inference is assumption-laden, the question is transparency — is underappreciated and important. The marketing consensus example (brand choice, switching costs) is well-chosen and genuinely demonstrates the structural approach at its best. The argument is less convincing as a general defense: many high-profile structural models have produced results that failed to hold up outside the estimation sample, and the paper does not engage seriously with the computational and identification challenges that have led practitioners toward quasi-experimental methods. The external-validation criterion is the right one but would also condemn many structural applications if applied consistently.