Panel Data & Fixed Effects

Python · from scratch + linearmodels  ·  R · plm, fixest  ·  Stock & Watson, 48 states × 7 years

Comparing a Unit to Itself

Repeated observations on the same units allow a different trick from anything in the previous groups: difference the confounders away. Subtracting each unit's own average from every one of its observations — the within transformation, and the estimator it produces is the within or fixed-effects estimator — leaves only variation over time inside a unit. If what contaminates a comparison is fixed within a unit — a state's geography, road network, drinking culture, the durable habits of its drivers — then comparing a unit to itself over time removes it, whether or not it was ever measured. No instrument required, and no claim that the confounders were observed.

Stock & Watson's traffic-fatality panel covers 48 states over 7 years, 1982–1988: 336 observations of the fatality rate and the state beer tax. It is the standard teaching example because the naive answer is so clearly wrong. Pooled across states, higher beer taxes go with more traffic deaths: a coefficient of +0.365.

That is state heterogeneity, not causation. States with severe drink-driving problems levy higher taxes because of them. The within (fixed-effects) estimator subtracts each state's own mean from every variable, so only within-state variation over time remains, and the entire cross-state comparison — where the confounding lives — is discarded.

Done from scratch and cross-checked against PanelOLS, the two agree exactly at −0.656. The sign flips, as the design predicts.

Where the Estimators Disagree

estimatorbeer-tax coefficientreading
pooled OLS+0.365confounded by state heterogeneity
within / fixed effects from scratch−0.656matches PanelOLS exactly
first difference+0.029opposite sign, indistinguishable from zero
random effects−0.052pulled back toward the pooled estimate

But the first-difference estimator does not agree, and that matters more than it first appears. Differencing consecutive years instead of subtracting means gives +0.029 — the opposite sign, and indistinguishable from zero.

Under strict exogeneity FD and FE are both consistent for the same parameter, so they should differ only by sampling noise. A sign flip is therefore diagnostic rather than incidental: it points to serial correlation in the errors, to dynamics a static specification omits, or to measurement error in the tax variable that differencing amplifies. Whichever it is, the assumption fixed effects leans on is not clean on this panel, and the FE number should not be read as though it were.

Random effects treats the state effect as a random draw uncorrelated with the regressors and quasi-demeans only partially, buying efficiency if that holds and bias if it does not. Here it returns −0.052, pulled almost all the way back to the confounded pooled estimate — the same omitted-variable contamination, differently packaged. The Hausman test rejects it (H=5.36H = 5.36, p=0.021p = 0.021), which is the formal instruction to use fixed effects.

Twoway Fixed Effects, and the Significance That Does Not Survive

State effects remove time-invariant state confounders but not shocks hitting every state at once — a recession, a federal seat-belt campaign, a fuel-price swing. Adding year effects absorbs those, giving the twoway specification that is the field standard:

frateit=βbeertaxit+αi+γt+εit\text{frate}_{it} = \beta\,\text{beertax}_{it} + \alpha_i + \gamma_t + \varepsilon_{it}
specificationestimateclustered SEp
one-way FE state only−0.6560.2890.024
twoway FE state + year−0.6400.3820.095
twoway FE + controls−0.5170.3520.144

Standard errors clustered by state, which is what accounts for within-unit serial correlation — without it the precision is badly overstated (Bertrand, Duflo & Mullainathan).

And this is where the result stops being comfortable. The coefficient barely moves — from −0.656 to −0.640 — but the pp-value goes from 0.024 to 0.095, and with covariates to 0.144.

Note why. The estimate is stable; it is the standard error that grows, because year effects absorb much of the variation the tax was competing to explain. The sign survives every within-state specification. The significance survives only the one that ignores national trends — and fatality rates plainly move with the business cycle, fuel prices and federal safety campaigns, so that is the specification with the weaker claim on us.

So what this panel defensibly supports is a negative sign whose magnitude is not pinned down. Taken with the first-difference disagreement, the beer-tax effect is real-looking but specification-dependent. That is a less quotable conclusion than "higher taxes save lives", and it is the one the numbers carry.

Two costs come with the method regardless. Fixed effects cannot identify time-invariant regressors — a state's geography, a person's sex — because those are swept out with the unit effect. And it requires the confounders to genuinely be time-invariant: a shock that changes a state differentially over time passes straight through.

Where this sits

The panel shape — units by time — is the same one modelled in the Multivariate Time Series and BVAR arc, where the target is forecasting the joint dynamics rather than isolating one coefficient. And the twoway specification here is difference-in-differences generalized to many groups and periods: the canonical two-group, two-period DiD is a twoway-FE regression with a single treatment indicator. The next page builds on it and confronts what happens when treatment timing varies across units, which is where the specification-dependence seen here finally acquires a name and a literature.

Notebooks

Downloads

References