Pitt-Shephard (1999) Filtering via Simulation: Auxiliary Particle Filters

Summary

This paper introduces the auxiliary particle filter (APF), an improvement to the generic particle filter (sequential Monte Carlo) for on-line inference in non-linear / non-Gaussian state-space models. In a state-space model with independent measurements $y_t\mid\alpha_t$ and a Markovian state $\alpha_t$ , filtering means recursively computing $f(\alpha_t\mid\mathcal F_t)$ via $f(\alpha_{t+1}\mid\mathcal F_{t+1})\propto f(y_{t+1}\mid\alpha_{t+1})\int f(\alpha_{t+1}\mid\alpha_t)\,dF(\alpha_t\mid\mathcal F_t)$ . A particle filter approximates the filtering density by a swarm of $M$ weighted "particles" (a random discrete support), propagating and reweighting them as data arrive. Pitt and Shephard diagnose the key weakness of the standard sampling/importance-resampling (SIR) filter — it degenerates when a new observation is an outlier or the likelihood is peaked in the tails of the prediction density — and fix it by introducing an auxiliary index variable that lets the filter "look ahead" to the next observation and resample before propagating, so particles are pushed into the region the new data favor. (Journal of the American Statistical Association 94(446): 590–599; also a chapter in Doucet–de Freitas–Gordon, Sequential Monte Carlo Methods in Practice.)

Key Claims

Particle filter = discrete random-support approximation. The continuous filtering variable $\alpha_t\mid\mathcal F_t$ is approximated by particles $\{\alpha_t^1,\dots,\alpha_t^M\}$ with masses $\{\pi_t^1,\dots,\pi_t^M\}$ ; treating this discrete support as the truth gives the empirical filtering density $\hat f(\alpha_{t+1}\mid\mathcal F_{t+1})\propto f(y_{t+1}\mid\alpha_{t+1})\sum_j \pi_t^j f(\alpha_{t+1}\mid\alpha_t^j)$ , from which new particles are drawn. As $M\to\infty$ the approximation converges.
The generic (SIR) filter and its weakness. The basic filter treats $\tfrac1M\sum_j f(\alpha_{t+1}\mid\alpha_t^j)$ as a prior, blindly propagates each particle through the transition, then reweights by the likelihood $f(y_{t+1}\mid\alpha_{t+1})$ and resamples. When the new measurement is surprising (outlier) or the likelihood is sharp, almost all propagated particles land where the likelihood is tiny, so a handful get nearly all the weight — sample impoverishment / weight degeneracy.
The auxiliary-variable fix. Augment the target with an auxiliary index $k$ identifying the parent particle, and perform a first-stage selection that looks ahead to the next observation — evaluating the likelihood at a representative point of each particle's predictive distribution (e.g. its mean/mode $\mu_{t+1}^j$ ) to pre-weight and resample parents before propagation. Only promising parents are propagated; a second-stage weight corrects for the approximation. This adapts the proposal to the incoming data.
Full adaptation as the ideal. A filter is fully adapted if it yields i.i.d. draws from the exact one-step target $f(\alpha_{t+1}\mid\mathcal F_{t+1})$ ; the APF approximates full adaptation cheaply, and is exactly fully adapted in tractable (e.g. conditionally Gaussian) cases.
Three ways to sample the one-step target. Given the mixture prior and an evaluable likelihood, one can draw from $f(\alpha_{t+1}\mid\mathcal F_{t+1})$ by sampling/importance resampling (SIR), acceptance (rejection) sampling, or MCMC — the APF is built on the importance-resampling route with the look-ahead auxiliary weights.
Refinements. Fixed-lag filtering and stratified sampling further reduce Monte Carlo variance; deliberately inducing negative correlation among particles (Carpenter–Clifford–Fearnhead) also helps.

Concepts Introduced or Extended

Entities Mentioned

Quotes

"Particle filters are the class of simulation filters which recursively approximate the filtering random variable $\alpha_t\mid\mathcal F_t$ by 'particles' … with associated discrete probability masses … a continuous variable is approximated by a discrete one with random support."

"Our approach is to extend the particle filter using an auxiliary variable … We will call a particle filter 'fully adapted' if it produces independent and identically distributed samples from [the one-step target]."

My Take

The auxiliary particle filter is the standard upgrade over the vanilla bootstrap/SIR filter, and its core idea — resample using a look-ahead at the next observation instead of after blindly propagating — is exactly what rescues particle filtering when the likelihood is informative or observations are occasionally extreme, which is the norm in stochastic-volatility and other financial state-space models. It complements the Kalman filter (exact only in the linear-Gaussian case) and the MCMC/data-augmentation smoothers (which process the whole sample off-line) by giving a genuinely on-line filter for non-linear, non-Gaussian models. The concept of full adaptation it introduced is now the yardstick by which particle-filter proposals are judged, and the paper sits directly upstream of the particle-MCMC and SV-estimation literature. Its cost is the extra look-ahead evaluation and the fact that, like all particle filters, performance still degrades in high state dimension.