Overview
Radford Neal is a Professor in the Departments of Statistics and Computer Science at the University of Toronto, where he holds a Canada Research Chair in Statistics and Machine Learning. He is one of the key figures in bringing Hamiltonian Monte Carlo from physics into statistics, and a pioneer in Bayesian neural networks.
Key Contributions / Features
- Hamiltonian Monte Carlo for statistics (Neal 1993, 1996a, 2011): introduced and popularized HMC for statistical applications, first for Bayesian neural network models. His 2011 handbook chapter is the canonical reference.
- Bayesian Learning for Neural Networks (Neal 1996a): established the theoretical and computational framework for fully Bayesian treatment of neural networks, using HMC as the primary sampler; showed that infinitely wide networks converge to Gaussian processes.
- Windowed HMC (Neal 1994): improved acceptance procedure using windows of states at trajectory endpoints to smooth discretization error.
- Tempered transitions (Neal 1996b): sampling from multimodal distributions by interpolating through a sequence of increasingly diffuse distributions within a single chain.
- Annealed importance sampling (Neal 2001): unbiased estimator of normalizing constants via importance-weighted tempering; foundational for modern normalizing flow estimators.
- Short-cut MCMC (Neal 2005, 2007): early termination of trajectories with unsuitable stepsizes to avoid wasted computation.
Related