Nobile (1995) A Hybrid Markov Chain for the Bayesian Analysis of the Multinomial Probit Model

Summary

National Institute of Statistical Sciences (NISS) Technical Report 36 (August 1995); published as Nobile (1998, Statistics and Computing 8: 229–242). Identifies a fundamental mixing failure in the McCulloch-Rossi (1994) Gibbs sampler for the multinomial probit (MNP) model: the MNP likelihood is constant along rays $(\beta, \Sigma) \to c(\beta, \Sigma)$ for $c > 0$, so the Gibbs chain moves efficiently across the iso-likelihood ridge but barely along it, leaving posterior inference heavily start-point dependent. The remedy is a hybrid Markov chain: after each complete Gibbs cycle, perform a Metropolis step that proposes rescaling the entire state by a scalar $c \sim F_C$. Because the candidate lies on the same iso-likelihood ray, the likelihood ratio in the Metropolis-Hastings (MH) acceptance probability equals 1 exactly — only prior densities need to be evaluated, making each rescaling step computationally trivial. Ergodicity of the hybrid chain is formally proved; simulated and real-data examples confirm dramatically faster mixing.

Key Claims

• Scale non-identification root cause: The MNP likelihood satisfies $\ell(c\beta, c^2\Sigma) = \ell(\beta, \Sigma)$ for all $c > 0$. Identification is supplied entirely by the prior (which implicitly enforces $\sigma_{11}=1$). The posterior is concentrated near the ray $\beta/\sqrt{\sigma_{11}} =$ const; the Gibbs sampler reaches this ray quickly but cannot travel efficiently along it.
• Hybrid Markov chain fix: After each Gibbs cycle over $(W, \beta, \Sigma)$, propose $\psi^* = c\psi'$ where $\psi = (\mathrm{vec}_L(S), \beta)$ and $S$ is the Cholesky factor of $\Sigma^{-1}$. The support of the candidate is the ray $S_{\psi'} = \{\psi^* : \psi^* = c\psi', c > 0\}$. Since $f(\psi^*)/f(\psi') = \pi(\theta^*)/\pi(\theta')$ (likelihood cancels), the MH acceptance probability reduces to a ratio of prior densities alone.
• Acceptance ratio in closed form: With prior $G \sim W_{p-1}(\nu, P)$ and $\beta|\Sigma \sim N_k(\beta_{(0)}, A_{(0)}^{-1})$, the ratio evaluates as: $$c^{-(p-1)(\nu+p/2)} \exp\!\left\{-\tfrac{1}{2}\!\left[(c^2-1)\beta'^T A_{(0)}\beta' - 2(c-1)\beta'^T A_{(0)}\beta_{(0)} + (c^{-2}-1)\,\mathrm{tr}\,P(S'S'^T)^{-1}\right]\right\}$$
• Four candidate distributions for $F_C$: (a) two-point mass mixture at $\{\gamma, \gamma^{-1}\}$; (b) mixture of uniforms on $(1,\gamma)$ and $(\gamma^{-1},1)$; (c) $c = U$ or $U^{-1}$, $U \sim \mathrm{Unif}(1,\gamma)$; (d) $\mathrm{Gamma}(\delta, \gamma)$. Recommended default: $c \sim \mathrm{Exp}(1)$.
• Ergodicity theorem (Appendix): The hybrid kernel $P_G P_M$ is (i) $\varphi$-invariant, (ii) $\varphi$-irreducible, (iii) aperiodic, (iv) absolutely continuous w.r.t. the posterior $\varphi$, (v) Harris recurrent. By Tierney (1994) Theorem 1, $\|(P_G P_M)^n(x,\cdot) - \varphi\| \to 0$ for all starting points.
• Easy case (Example 1): Binomial probit ($\beta=-2$, $\sigma=2$, $n=2000$): Gibbs and hybrid give similar posteriors — the mixing failure is mild at moderate signal-to-noise.
• Hard case (Example 2): Binomial probit ($\beta=5$, $\sigma=2$, $n=2000$): two Gibbs chains from $(\beta,\sqrt{\sigma})=(5,\sqrt{2})$ and $(25,5)$ yield completely different posterior histograms for $\beta/\sqrt{\sigma}$ after 20,000 draws. Hybrid chain yields nearly identical histograms from both starts.
• Real data (Example 3): Dutch mobility panel, three-alternative car-ownership model: hybrid reaches stationarity quickly across all parameters; Gibbs still shows trending in the correlation $\rho_{12}=\sigma_{12}/\sqrt{\sigma_{11}\sigma_{22}}$ after 20,000 draws.

Concepts Introduced or Extended

• Multinomial Probit
• Gibbs Sampler
• Metropolis-Hastings Algorithm
• Data Augmentation
• Discrete Choice Models

Entities Mentioned

• Agostino Nobile
• Robert E. McCulloch
• Peter E. Rossi
• James H. Albert
• Siddhartha Chib

Quotes

"The idea is very simple: after each Gibbs cycle through the full conditional distributions of $W$ and $\theta$, perform a Metropolis step to change the scale of the current state. This allows the chain to move faster across the parameter/latent data space."

My Take

The paper correctly diagnoses a genuine and practically important failure of the McCulloch-Rossi Gibbs sampler: scale non-identification of the MNP likelihood creates a near-degenerate posterior ridge that standard Gibbs cannot traverse. The hybrid chain is an elegant minimal fix — the rescaling step costs essentially nothing per iteration because the likelihood ratio cancels. The ergodicity proof is clean and complete.

Note on the published version: Imai and van Dyk (2005) show that Nobile (1998) contains an error in the Metropolis acceptance probability: substituting $(\tilde\beta, \tilde\Sigma)$ for $(\beta, \Sigma)$ distorts the stationary distribution by a factor $c^{-(2-k-p(p-1)/2)}$. Imai-van Dyk correct the algorithm and reinterpret the rescaling step as a marginal data augmentation (MDA) move — the conceptual frame that drives their far more efficient Algorithm 1. The 1995 NISS technical report may not contain the same error; the acceptance ratio expression on p. 10 appears formally correct as written.