Non-Centered Parameterization

Definition

A non-centered parameterization rewrites a hierarchical model so that lower-level ("local") parameters are drawn from a fixed, standardized distribution and made deterministic functions of the higher-level ("global") parameters, rather than being drawn directly from a distribution whose location and scale are the global parameters. For a normal group effect the centered form is $\theta_i \sim \mathcal N(\mu, \tau)$; the non-centered form is $$\theta_i = \mu + \tau\,\eta_i, \qquad \eta_i \sim \mathcal N(0,1),$$ so the sampled variables $\eta_i$ are a priori independent of $(\mu,\tau)$. The choice of parameterization does not change the model — only the geometry of the posterior the sampler must explore.

Key Ideas

• The funnel. With sparse or weak data, the centered hierarchy produces a funnel-shaped posterior: as the group scale $\tau=e^{v/2}$ shrinks, the $\theta_i$ collapse into a narrow neck of high density but low volume; as $\tau$ grows, a wide mouth of low density but high volume opens. The two regions carry comparable probability mass and the curvature varies by orders of magnitude (Betancourt-Girolami 2015; Neal 2003).
• Local, not global, correlation. The pathological correlations are position-dependent, so a single global rotation or mass-matrix correction cannot remove them — the reason Gibbs and Random-Walk Metropolis stall and even naively-tuned HMC struggles.
• Weak data → non-center; strong data → center. The non-centered form flattens the geometry when the likelihood is weak (the prior dominates the local–global link); the centered form is better when the data are informative about the locals. No single parameterization is uniformly optimal (Papaspiliopoulos–Roberts–Sköld 2007).
• Interpolation. Partially non-centered parameterizations interpolate between the two extremes and can be tuned to the strength of the data.

How It Works

Sampling the non-centered model draws $\eta_i \sim \mathcal N(0,1)$ and the globals $(\mu,\tau)$, then reconstructs $\theta_i = \mu + \tau\eta_i$ for interpretation. Because $\eta_i \perp (\mu,\tau)$ under the prior, the joint posterior over $(\eta, \mu, v)$ is close to isotropic when the data are weak, so a sampler tuned to marginal scales explores efficiently. Under a strong likelihood the data re-introduce dependence, and the centered form — where $\theta_i$ is directly informed — mixes better.

For gradient-based samplers (HMC/NUTS), the practical recipe is: reparameterize non-centered for weakly-identified groups, run NUTS, and monitor divergences — divergent leapfrog trajectories flag regions (typically the funnel neck) where the fixed integrator step size cannot resolve the curvature, indicating biased exploration. Riemannian-manifold HMC (a position-dependent metric) is the alternative that adapts to local curvature directly.

Why It Matters

• Non-centering is the single most common fix that makes hierarchical and multilevel models sample-able in modern probabilistic programming (Stan, PyMC).
• It reframes a computational failure (poor mixing, divergences) as a geometry problem solvable by reparameterization rather than by throwing more iterations at the sampler.
• The same centered/non-centered idea recurs in data augmentation and parameter-expansion methods (Gelman–Van Dyk; Liu–Wu), where auxiliary-variable scaling changes the augmented-posterior geometry.

Open Questions

• Automatic parameterization. Choosing (or interpolating) the parameterization per group from the data, rather than by hand, remains an active problem.
• Beyond the normal case. Non-centering is cleanest for location–scale families; general hierarchical likelihoods may lack a convenient standardizing transform.

Related

• Hamiltonian Monte Carlo
• Bayesian Hierarchical Model
• Multilevel Model
• Data Augmentation
• Markov Chain Monte Carlo
• Betancourt-Girolami (2015)
• Michael Betancourt
• Mark Girolami