Geweke-Tanizaki (2001) Bayesian Estimation of State-Space Models Using the Metropolis-Hastings Algorithm within Gibbs Sampling

Summary

This paper gives a general Markov-chain-Monte-Carlo solution to estimating nonlinear and/or non-Gaussian state-space models in a Bayesian framework. It combines the Gibbs sampler with the Metropolis-Hastings algorithm to draw the latent state at each date from its full conditional given the neighboring states and the observation, so the measurement and transition equations may take any form and the errors need not be normal. The scheme extends the Gibbs-sampling state-space work of Carlin-Polson-Stoffer (1992) and Carter-Kohn (1994, 1996) — which required conjugate/easily-sampled conditionals or rejection sampling — and delivers an asymptotically exact estimate of the smoothing mean; the authors compare several proposal-density choices for the Metropolis step.

Key Claims

• The goal: general nonlinear/non-Gaussian smoothing. Given a state-space model $y_t=h_t(\alpha_t,\varepsilon_t)$, $\alpha_t=f_t(\alpha_{t-1},\eta_t)$ with arbitrary functions and non-normal errors, obtain the smoothing distributions $p(\alpha_t\mid y_{1:T})$ and their means — infeasible in closed form outside the linear-Gaussian (Kalman) case.
• Metropolis-within-Gibbs on the states. Sample each state $\alpha_t$ from its full conditional $p(\alpha_t\mid\alpha_{t-1},\alpha_{t+1},y_t)$ (a single-move sweep) using a Metropolis-Hastings step, since that conditional is known only up to proportionality for general $h_t,f_t$. Cycling over $t$ yields joint draws of the whole state path.
• Why not the alternatives. Carter-Kohn / Carlin-Polson-Stoffer rely on conjugacy or rejection sampling, which is computationally inefficient (or fails, when the acceptance probability is near zero) for genuinely nonlinear/non-Gaussian systems; particle resampling smoothers (Gordon-Salmond-Smith 1993; Kitagawa 1996) are accurate but very slow for smoothing.
• Proposal density matters. The precision and mixing of the MH step depend on the proposal; the paper evaluates several candidates (e.g., drawing from the transition prior versus tailored proposals) and their effect on the accuracy of the estimated smoothing mean.
• Asymptotically exact. Unlike the deterministic numerical-integration filters (Kitagawa 1987) or importance-sampling smoothers, the MCMC estimate of the smoothing mean is exact in the limit of infinitely many draws.

Concepts Introduced or Extended

• State-Space Representation
• Metropolis-Hastings Algorithm
• Gibbs Sampler

Entities Mentioned

• John Geweke
• Hisashi Tanizaki

Quotes

"Using the Gibbs sampler and the Metropolis-Hastings algorithm, an asymptotically exact estimate of the smoothing mean is obtained from any nonlinear and/or non-Gaussian model."

My Take

The paper sits at the moment when state-space estimation split into two lineages: the sequential-Monte-Carlo (particle) route and the MCMC route, and this is a clean statement of the MCMC single-move approach for the fully general model. Its pragmatic virtue is generality — by putting a Metropolis step inside the Gibbs sweep over states, it removes the conjugacy/rejection requirements that made Carter-Kohn elegant but restrictive. The cost, well known in hindsight, is that single-move samplers mix slowly when the states are highly persistent (adjacent $\alpha_t$ are strongly correlated), which is exactly why the later literature moved to block/forward-filtering-backward-sampling updates and to the mixture-approximation tricks used for stochastic volatility. For the wiki it is the bridge from the linear-Gaussian Kalman filter and the particle filter to general Bayesian state-space estimation, and a companion to the state-space page's Tanizaki (1993) qualitative-data material.