Definition
Charter school effectiveness research estimates the causal effect of charter school attendance on student academic outcomes — primarily standardized test scores — relative to the counterfactual of attending a traditional public school. Because charter schools attract applicants who may differ systematically from non-applicants (ability, motivation, parental engagement), simple comparisons are confounded. The dominant identification strategy uses oversubscription lotteries: when a charter school receives more applications than available seats, it holds a random lottery, generating as-good-as-random variation in attendance among otherwise comparable applicants.
Key Ideas
- Lottery-based instrumental variable (IV): Lottery offer is an instrument for charter attendance. Compliers are applicants whose enrollment is determined by the lottery outcome (not those who would always or never attend). The local average treatment effect (LATE) identified is the causal effect for this marginal applicant group — typically lower-achieving, lower-income urban students who actively sought a charter seat.
- No Excuses model: A dominant strand of high-performing urban charter schools (KIPP, Success Academy, Uncommon Schools, YES Prep) shares a common bundle of inputs: extended instructional time (10+ hour days, longer school year), strict behavioral expectations and discipline codes, intensive focus on math and reading, data-driven instruction, selective and highly motivated teachers, and binding commitment contracts from students and parents.
- Heterogeneous effects: Average effect sizes mask important variation. Reading gains in particular tend to be concentrated among students with the greatest academic need — Limited English Proficiency (LEP) students, special education (SPED) students, and low baseline achievers — while math gains are typically more broad-based.
- Sorting critique: Critics argue high-performing charter schools achieve results by attracting motivated families and excluding difficult-to-serve students (LEP, SPED, English learners). Lottery-based evidence from KIPP Lynn (Angrist et al. 2010) directly contradicts this for at least some schools: KIPP Lynn applicants start below the state mean and include high proportions of LEP and SPED students, yet achieve large gains.
How It Works
Identification via oversubscription lottery:
- School receives more applications than seats → random lottery determines admission offers
- Lottery offer (intent-to-treat) is as-good-as-random conditional on application year and entry grade
- IV estimate converts intent-to-treat (lottery offer) into treatment-on-treated (actual attendance) using two-stage least squares
- Complier population: those whose attendance decision is driven by the lottery outcome — typically disadvantaged applicants who could not secure alternatives
Angrist et al. (2010) — KIPP Lynn, Massachusetts:
- Grades 5–8 middle school; predominantly Hispanic; high LEP and SPED shares
- Applicants enter 4th grade ~0.3σ below state mean in math
- Causal effects: +0.35σ per year in math; +0.12σ per year in reading
- Reading heterogeneity: gains of ~0.30–0.35σ for LEP and SPED students; near-zero for others
- Math gains broad-based across the student distribution
- Inputs examined: 10.5-hour day, August start, Saturday school, strict behavioral code, commitment contracts
Why It Matters
- Evidence base for policy: Lottery-IV studies provide the most credible causal evidence on whether charter schools improve student outcomes, replacing prior observational work that was heavily contested.
- Cream-skimming debate: Rigorous lottery evidence from high-performing No Excuses schools shows gains concentrated among the most disadvantaged students, directly refuting the most common critique of charter expansion.
- Input identification: Establishing that the No Excuses bundle produces large achievement gains motivates the next research question: which specific inputs (time, teachers, culture, curriculum) are responsible? This is the agenda pursued by Dobbie and Fryer (2013).
- Scale: KIPP is the largest charter management organization in the US; evidence on KIPP effectiveness directly informs debates about charter school expansion and replication.
Open Questions
- Which inputs drive effects? Angrist et al. (2010) estimate the effect of the full No Excuses bundle but cannot decompose it. Subsequent work (Dobbie and Fryer 2013) attempts to identify specific school practices, but clean causal separation of inputs within the bundle remains difficult.
- External validity: KIPP Lynn is one school in one city. Generalizability to the broader KIPP network and to other charter models (virtual charters, selective charters, conversion charters) is not established by a single lottery study.
- Long-run effects: Most lottery-IV studies measure gains in test scores over the years of enrollment. Effects on high school graduation, college enrollment, and adult earnings are estimated in a smaller literature and are less well-established.
- Mechanisms for LEP/SPED heterogeneity: Why does the No Excuses model produce particularly large reading gains for LEP and SPED students? Extended time, intensive literacy instruction, and structured behavioral environments are plausible channels, but the mechanism is not identified within these studies.
Related
Sources