McCulloch-Polson-Rossi (2000) A Bayesian Analysis of the Multinomial Probit Model with Fully Identified Parameters

Summary

McCulloch, Polson, and Rossi (2000) develop the "ID prior" — a prior placed directly on the identified parameter space of the multinomial probit (MNP) model — as an alternative to the McCulloch-Rossi (1994) "NID" approach, which works in the unidentified space and reports marginal posteriors. The key device is reparameterising the covariance matrix $\Sigma$ so that $\sigma_{11} = 1$ with probability one, then assigning independent Normal and inverse-Wishart priors to the resulting free parameters. The posterior is computed by a four-block Gibbs sampler with no tuning parameters. The gain is interpretable, informative priors on the identified coefficients; the cost is higher chain autocorrelation relative to the NID sampler.

Key Claims

• Identification: Parameters $(\beta, \Sigma)$ are not identified because $Y(cW) = Y(W)$ for all $c > 0$; fixing $\sigma_{11} = 1$ achieves identification. NID approach (McCulloch-Rossi 1994) uses a proper prior on the full space and marginalises; ID approach enforces the constraint in the prior itself.
• ID reparameterisation: Write $\varepsilon = (U, Z')'$ where $U = \varepsilon_1$, $Z = (\varepsilon_2,\ldots,\varepsilon_{p-1})'$. Define $\gamma = E(UZ)$ and $\Phi = \Sigma_Z - \gamma\gamma'/\sigma_{11}$ (conditional variance of $Z|U$, the Schur complement). Then $\Sigma = \begin{bmatrix} 1 & \gamma' \\ \gamma & \Phi + \gamma\gamma' \end{bmatrix}$ and the one-to-one map $(\sigma_{11}, \gamma, \Phi) \leftrightarrow \Sigma$ allows a prior on $\{\Sigma \mid \sigma_{11} = 1\}$ via $\gamma \sim N(\bar\gamma, B^{-1})$ and $\Phi^{-1} \sim W(\kappa, C)$.
• ID Gibbs sampler (four blocks, all conjugate, no tuning):
  1. $\beta \mid \gamma, \Phi, \{W_i\}, D \sim$ multivariate Normal
  2. $W_{ij} \mid \beta, \gamma, \Phi, \{W_{i(-j)}\}, D \sim$ truncated univariate Normal
  3. $\gamma \mid \Phi, \beta, \{W_i\}, D \sim$ Normal (regression of $Z$ on $U$)
  4. $\Phi \mid \gamma, \beta, \{W_i\}, D \sim$ inverted Wishart
• Analytical result (NID prior, $\beta$): Under the NID prior $\tilde\beta \sim N(\bar b, A^{-1})$, the identified $\beta = \tilde\beta/\sqrt{\tilde\sigma_{11}}$ follows $\sqrt{v^{1.2}}\,\chi \cdot \tilde\beta$ where $\chi^2 \sim \chi^2_{\nu-p+2}$; heavy-tailed when $\bar b=0$, skewed when $\bar b\neq0$. $\beta$ and $\Sigma$ are not independent under the NID prior.
• Analytical result (NID prior, $\gamma$): Under the NID prior, the identified $\gamma = \sigma_{21}$ follows a multivariate-$t$ with $\nu-p+3$ degrees of freedom, $E(\gamma) = (V^{22})^{-1}v^{21}$, $\text{Var}(\gamma) = v^{1.2}(\nu-p+1)^{-1}(V^{22})^{-1}$.
• Analytical result (NID prior, $\Phi$): $\Phi^{-1} = W/\omega$ where $W \sim \text{Wishart}(\nu, (V^{22})^{-1})$ and $\omega \sim v^{1.2}\chi^2_{\nu-p+2}$, independent.
• Prior assessment for $E(\Sigma) = I$: Setting $E(\gamma) = 0$ and $\Delta = I$ reduces the ID prior to two scalar choices: $\kappa$ (Wishart d.f.) and $\tau$ (variance of $\gamma$); large $\kappa$ and small $\tau$ tighten the prior toward $I$. Recommended default: $\kappa = p+2$, $\tau = 1/8$.
• Mixing trade-off: ID sampler autocorrelations are higher than NID. More diffuse $\Phi$ prior $\to$ slower mixing. In $p=3$ example, lag-10 autocorrelation of $\sigma_{22}$ is 0.71 (proper ID) vs. 0.77 (improper ID) vs. lower for NID. Improper $\Phi$ prior can cause sampler to get stuck in $p=6$ case when likelihood is not informative enough.
• Eigenvalue pathology: The improper ($\kappa=0$) $\Phi$ prior acts as a highly informative prior on the smallest eigenvalue of $\Sigma$, pushing it toward zero. Confirmed by simulating from $N_5(0,I)$ with $n=7$ observations: posterior of smallest eigenvalue is dominated by prior (peaks near 0.02) despite near-identity truth.
• Advantages over NID: (a) Truly informative priors on $\beta$ are straightforward (just $N(\bar\beta,D^{-1})$); (b) flat improper prior on $\beta$ is allowed; (c) naturally accommodates hierarchical models $\beta_j \sim N(\bar\beta, V_\beta)$ since priors on identified parameters are interpretable; (d) analytical prior assessment results are available (unique to this parameterisation).

Concepts Introduced or Extended

• Multinomial Probit
• Identification
• Gibbs Sampler
• Covariance Matrix Estimation

Entities Mentioned

• Robert E. McCulloch
• Nicholas G. Polson
• Peter E. Rossi

Quotes

"Our new approach places a prior directly on the identified parameter space. The key is the specification of a prior on the covariance matrix so that the (1,1) element is fixed at 1 and it is possible to draw from the posterior using standard distributions." (Abstract)

"The more diffuse the prior on $\Sigma$ is, the slower the autocorrelations die out." (p. 11)

My Take

The paper's main contribution is resolving a genuine awkwardness in McCulloch-Rossi (1994): when the prior is placed on the unidentified space, the induced prior on the identified parameters has an unusual form ($\sqrt{\chi^2} \times$ Normal), making informative prior specification opaque. The ID approach trades this opacity for slower mixing — a reasonable engineering trade-off when the researcher has meaningful prior information (e.g., in a hierarchical model). The eigenvalue analysis in Section 7 is particularly useful: it explains concretely why the improper $\Phi$ prior is dangerous and provides a diagnostic (simulate with tiny n, inspect posterior of smallest eigenvalue). The paper has limited empirical content; the Imai-van Dyk (2005) marginal augmentation approach later showed that the NID chain can be dramatically accelerated, partially closing the gap.