Neal (2011) MCMC Using Hamiltonian Dynamics

Summary

A comprehensive handbook chapter (Chapter 5 of Handbook of Markov Chain Monte Carlo, Brooks–Gelman–Jones–Meng eds., CRC 2011) that serves as the canonical statistical reference for Hamiltonian Monte Carlo (HMC). Neal derives the algorithm from first principles, explains three key properties of Hamiltonian dynamics that make it useful for MCMC (reversibility, volume preservation, approximate energy conservation), presents the leapfrog integrator in detail, discusses tuning and scaling theory, and surveys five variations — windowed HMC, partial momentum refreshment, approximate trajectory computation, short-cut methods, and tempered trajectories — plus connections to other MCMC families.

Key Claims

• HMC avoids random-walk diffusion by simulating trajectories under Hamiltonian dynamics; proposals are distant but accepted with high probability because energy $H = U(q) + K(p)$ is approximately conserved.
• The leapfrog (Störmer-Verlet) integrator is symplectic: it exactly preserves phase-space volume even in discrete form, so the Jacobian of the proposal map is 1 and no correction term appears in the Metropolis acceptance probability.
• Reversibility is obtained by negating the momentum at the end of a trajectory; detailed balance then holds for the augmented $(q, p)$ space.
• Optimal acceptance rate for HMC is approximately 65% (Roberts–Gelman–Gilks 1997), achieved by tuning the leapfrog stepsize $\varepsilon$; number of steps $L$ is tuned so the trajectory length $\varepsilon L$ is comparable to the autocorrelation length.
• Scaling with dimensionality: random-walk Metropolis requires $O(d)$ proposals per independent sample; HMC requires $O(d^{1/4})$. Empirically (100-dimensional Gaussian), HMC needed ~10× fewer gradient evaluations than random-walk Metropolis for the same estimation error.
• Windowed HMC (Neal 1994) improves acceptance by averaging probability densities over a window of states at the start and end of each trajectory, smoothing out oscillations in $H$ caused by leapfrog discretization error.
• Partial momentum refreshment (Horowitz 1991) makes successive trajectories correlated in direction, further reducing autocorrelation without sacrificing detailed balance.
• Tempered trajectories — multiply momentum by $\alpha > 1$ in the first half, divide by $\alpha$ in the second half — allow the chain to cross low-probability barriers between modes.
• As of 2011, HMC "seems to be under-appreciated by statisticians"; the No-U-Turn Sampler (NUTS, Hoffman–Gelman 2011) and Stan subsequently made automatic HMC mainstream.

Concepts Introduced or Extended

• Hamiltonian Monte Carlo
• Markov Chain Monte Carlo
• Metropolis-Hastings Algorithm

Entities Mentioned

• Radford M. Neal

Quotes

"Hamiltonian dynamics can be used to produce distant proposals for the Metropolis algorithm, thereby avoiding the slow exploration of the state space that results from the diffusive behaviour of simple random-walk proposals."

"Despite the large overlap in their application areas, the MCMC and molecular dynamics approaches have continued to co-exist in the following decades."

My Take

The definitive entry point for HMC in the statistics literature. The exposition is unusually careful — Neal proves every claim (volume preservation, reversibility, detailed balance) from scratch rather than citing physics texts. The scaling argument (Section 4.4) is the clearest justification for why HMC dominates random-walk methods in high dimensions. The 2011 timing is notable: this chapter appeared just as Stan launched with HMC/NUTS, making it the theoretical companion to modern probabilistic programming. One limitation: the paper predates NUTS and the No-U-Turn criterion, so the trajectory-length tuning discussion is somewhat manual; in practice NUTS has superseded the windowed and short-cut variants Neal describes.