Geweke (1992) Evaluating the Accuracy of Sampling-Based Approaches to the Calculation of Posterior Moments

Summary

Puts Markov chain Monte Carlo (MCMC) numerical accuracy and convergence on formal footing using spectral analysis. The chain output $\{g(\theta^{(t)})\}$ is viewed as a stationary time series; the variance of the MCMC posterior mean estimate is $S_0/n$ where $S_0$ is the spectral density at frequency zero. Two diagnostic tools follow: the Numerical Standard Error (NSE), which is the properly-corrected standard error of any posterior mean estimate; and the Geweke z-score, a formal convergence test comparing the means of early vs. late chain segments.

Key Claims

• Numerical Standard Error (NSE). The variance of the MCMC average $\hat{E}[g(\theta)] = n^{-1}\sum_{t=1}^n g(\theta^{(t)})$ is $S_0/n$ where $S_0$ is the spectral density of $\{g(\theta^{(t)})\}$ at frequency zero, capturing all serial correlation in the chain. NSE $= \sqrt{S_0/n}$ is the correct standard error, larger than the naive $\hat\sigma_g/\sqrt{n}$ by a factor of $\sqrt{S_0/\hat\sigma_g^2}$. Geweke recommends estimating $S_0$ via an autoregressive (AR) approximation with Bayesian information criterion (BIC) lag selection.
• Relative Numerical Efficiency (RNE). RNE $= \hat\sigma_g^2/S_0 \leq 1$ is the ratio of i.i.d. efficiency to MCMC efficiency. RNE $< 1$ reflects the efficiency cost of serial correlation; RNE $= 1$ means the chain is as efficient as i.i.d. sampling.
• Geweke convergence diagnostic (z-score). Partition the chain into an early segment (first $n_a$ iterations, typically 10%) and a late segment (last $n_b$ iterations, typically 50%), discarding the middle. Under convergence both segments estimate the same posterior mean. The test statistic $z = (\bar{g}_A - \bar{g}_B)/\sqrt{\hat S_{0,A}/n_a + \hat S_{0,B}/n_b} \overset{d}{\to} N(0,1)$ is the Geweke z-score. Rejection suggests the chain has not converged.
• Stationarity assumption. The spectral approach assumes the chain has reached its stationary distribution; the convergence diagnostic tests this assumption informally by checking whether early and late means agree. It does not guarantee convergence but flags obvious failure.
• Applications. Illustrated on the normal linear model (seemingly-unrelated-regression, SUR, style) and on Markov mixture models, showing that chains for mixture parameters can have very low RNE due to high serial correlation in the component membership indicators.

Concepts Introduced or Extended

• MCMC Convergence Diagnostics
• Gibbs Sampler

Entities Mentioned

• John Geweke

Quotes

"Data augmentation and Gibbs sampling are two closely related, sampling-based approaches to the calculation of posterior moments. The fact that each produces a sample whose constituents are neither independent nor identically distributed complicates the assessment of convergence and numerical accuracy."

My Take

The foundational paper for MCMC numerical accuracy. The NSE and Geweke z-score remain standard diagnostics in the coda and posterior R packages three decades later. The key conceptual contribution is recognising that MCMC output is a stationary time series and that spectral methods — specifically the zero-frequency spectral density — provide the right tool for variance estimation. The RNE concept is useful but understated: it quantifies exactly the efficiency penalty of correlated sampling, connecting MCMC to the classical effective sample size ($n_{\text{eff}} = n \times \text{RNE}$). Limitation: the convergence diagnostic is a necessary but not sufficient condition for convergence; multimodal posteriors can fool it entirely.