Definition
Count-data regression models a non-negative integer response y∈{0,1,2,…} (event counts) as a function of covariates. The baseline is Poisson regression, a generalized linear model (GLM) with a log-linear mean logμ=x⊤β and the Poisson variance restriction Var(y)=μ. Because real count data — especially in economics and the social sciences — routinely violate that restriction through over-dispersion (Var(y)>μ) and an excess of zeros, a ladder of richer models relaxes it: quasi-Poisson, negative binomial, and the zero-augmented hurdle and zero-inflated models.
Key Ideas
- Two failures of plain Poisson. (1) Over-dispersion: the variance exceeds the mean, so Poisson standard errors are too small. (2) Excess zeros: far more zero counts than any Poisson/NB fit implies.
- A shared mean, different likelihoods. The models below all keep the log-linear mean logμ=x⊤β; they differ only in how they model dispersion and the probability of a zero.
- Two orthogonal fixes. Over-dispersion is handled by dispersion adjustment or an extra shape parameter (quasi-Poisson, NB); excess zeros are handled by a second model component (hurdle or zero-inflation). A model can need both.
- Hurdle ≠ zero-inflation. Both add a component for zeros but tell different stories: hurdle treats all zeros as a single "didn't cross the hurdle" event; zero-inflation splits zeros into structural (always zero) and sampling (a count that happened to be zero) types.
How It Works
The over-dispersion ladder.
- Poisson GLM — y∣x∼Poisson(μ), logμ=x⊤β, estimated by ML. Assumes Var=μ.
- Quasi-Poisson — same mean, a free dispersion parameter ϕ with Var=ϕμ, estimated by quasi-ML / GEE; "adjusted Poisson" instead keeps Poisson estimates but uses sandwich standard errors.
- Negative binomial (NB) — an extended GLM with an extra shape parameter θ giving Var=μ+μ2/θ (a gamma-mixed Poisson), estimated by ML; captures over-dispersion within a full likelihood.
The excess-zeros extensions (zero-augmented models).
- Hurdle model (Mullahy 1986) — a two-part model: a binary component for zero-vs-positive (the "hurdle", a right-censored count/binary process) and a zero-truncated count model (Poisson or NB) for the positive counts. The two parts may use different covariates and are estimated separately.
- Zero-inflated model (Lambert 1992) — a mixture of a point mass at zero (with probability π(z), typically a logit in covariates z) and a count distribution (Poisson → ZIP, NB → ZINB) that itself can produce zeros. So P(y=0)=π+(1−π)f(0).
- Sample selection for counts (Greene 1994) — the count is observed only for a selected subsample, and if the selection error correlates with the count's unobservables, naive estimates are biased. Greene ports the Heckman selection correction from continuous outcomes to Poisson/NB counts (a probit selection equation jointly modelled with the count), extending the selection model to count data.
Why It Matters
- Ubiquity. Counts (doctor visits, patents, accidents, roll-call events, defaults) are everywhere in applied micro-econometrics and the social sciences, and the Poisson restriction almost never holds.
- Substantive zero mechanisms. Choosing hurdle vs. zero-inflation is a modeling statement about why zeros occur (a single participation gate vs. a mix of never-users and occasional-zero users), not just a goodness-of-fit tweak.
- Standard tooling. Zeileis-Kleiber-Jackman (2008) gave these models a base-
glm()-compatible implementation (pscl::hurdle, zeroinfl), making them routine in applied work; Cameron–Trivedi (1998) is the econometrics reference.
Open Questions
- Model choice among the zero-augmented options (hurdle vs. ZIP vs. ZINB) is often driven by fit criteria (AIC/Vuong tests) rather than a clear theory of the zero process; the Vuong test for non-nested comparison is itself debated.
- Identification of the two components can be weak when the same covariates drive both the count and the zero parts.
- Over-dispersion vs. excess zeros are entangled — excess zeros induce apparent over-dispersion, so disentangling which mechanism is operating is not always clean. Greene (1994) warns that excess zeros can masquerade as over-dispersion and proposes a test to distinguish zero inflation (ZIP/ZINB) from genuine over-dispersion (NB).
Related