Chib-Greenberg (1998) Analysis of Multivariate Probit Models

Summary

Provides a unified simulation-based framework for Bayesian and non-Bayesian analysis of the multivariate probit model. A three-block Markov chain Monte Carlo (MCMC) sampler handles the latent data, regression parameters, and correlation matrix; the correlation block requires Metropolis-Hastings (M-H) with a tailored proposal density because the full conditional has no conjugate form. The Chib (1995) marginal likelihood identity is extended to settings where some full conditionals have unknown normalising constants, requiring kernel density estimation for the posterior ordinate of the correlation parameters. Three empirical applications compare correlated, equi-correlated, and independent probit specifications via Bayes factors.

Key Claims

• Model: $Y_{ij} = I(z_{ij}>0)$ where $Z_i \sim N_J(X_i\beta, \Sigma)$; $\Sigma$ must be in correlation form (unit diagonal) — the covariance parameterisation is not likelihood-identified. $p = J(J-1)/2$ free parameters $\sigma \equiv (\sigma_{12},\ldots,\sigma_{J-1,J})$.
• Block 1 — latent data $Z_i|y_i,\beta,\Sigma$: multivariate normal (MVN) truncated to region $B_i$ (product of half-lines determined by $y_i$); sampled by cycling through $J$ univariate truncated normals (Geweke 1991 method; inverse cumulative distribution function, CDF).
• Block 2 — regression parameters $\beta|Z,\Sigma$: conjugate Normal (eq. 6) — $\hat\beta = B^{-1}(B_0\beta_0+\sum_i X_i'\Sigma^{-1}Z_i)$, $B = B_0 + \sum_i X_i'\Sigma^{-1}X_i$.
• Block 3 — correlation parameters $\sigma|y,Z,\beta$: M-H with tailored proposal (eq. 7): $\sigma' = \mu^* + h$, $\mu^* = \mu + P(\sigma-\mu)$, where $\mu$ = mode of $\log g(\sigma|Z,\beta)$ via Newton-Raphson, $V$ = negative inverse Hessian, $h\sim t_\nu(0,\tau^2 V)$; $P=0$ gives independence chain, $P=-I_p$ gives reflection chain; for large-dimensional $\sigma$, partition into blocks applied sequentially.
• Marginal likelihood — extension of Chib (1995) (eq. 8): $\log m(y) = \log\text{pr}(y|\beta^*,\Sigma^*) + \log\pi(\beta^*) + \log\pi(\sigma^*) - \log\hat\pi(\beta^*|y,\Sigma^*) - \log\hat\pi(\sigma^*|y)$; the $\beta$ ordinate uses Rao-Blackwellisation (eq. 9) from a reduced run fixing $\Sigma^*$; the $\sigma$ ordinate uses kernel density estimation (eq. 10, bandwidth $b_j=s_jG^{-1/(p+4)}$) because the normalising constant of $\pi(\sigma|Z,\beta)$ is unknown; for high-dimensional $\sigma$: sequential block decomposition $\pi(\sigma^*|y)=\pi_1(\sigma_1^*|y)\pi_2(\sigma_2^*|y,\sigma_1^*)$ each estimated from reduced runs.
• Likelihood contribution $\log\text{pr}(y|\beta^*,\Sigma^*)$: Geweke-Hajivassiliou-Keane (GHK) method — Cholesky $\Sigma^*=LL'$, change of variables $Z_i=X_i\beta+L\varepsilon_i$, recursive truncated-normal draws (eq. 11), averaged over 10,000 Monte Carlo iterations.
• Prior on $\sigma$ normalising constant: MC proportion of draws from $\phi_p(\sigma|\sigma_0,G_0^{-1})$ satisfying positive definiteness constraint.
• MCEM (maximum likelihood estimation, MLE): Monte Carlo expectation-maximization (EM; Wei-Tanner 1990) — E-step approximated by average of $\log f(Z|\theta)$ over $N$ draws of $Z$; M-step: closed-form $\beta$ update, Newton-Raphson $\sigma$ update; standard errors via Louis (1982) observed information.
• Application 1 — voter behaviour ($n=95$, $J=2$, Troy Michigan): correlated vs. independent probit; $\log m = -126.31$ vs. $-126.30$; Bayes factor (BF) $\approx 1$ — no evidence for correlation.
• Application 2 — Six Cities wheezing ($n=537$, $J=4$, child wheeze ages 7–10 vs. mother's smoking): $\mathcal{M}_1$ unrestricted (6 free $\sigma$), $\mathcal{M}_2$ equi-correlated, $\mathcal{M}_3$ independent; $\log m = -825.03$, $-816.80$, $-931.16$; BF$_{2,3}=4.63\times10^{49}$ — decisive against independence, equi-correlated preferred over unrestricted.
• Application 3 — Panel Study of Income Dynamics (PSID) labour force participation ($n=520$, $J=7$, married women 35–62, 1976–82): $\mathcal{M}_1$ unrestricted (21 free $\sigma$, M-H in 4 blocks of 6/6/6/3), $\mathcal{M}_2$ equi-correlated; $\log m = -1561.66$ vs. $-1600.45$; decisive evidence for unrestricted model.

Concepts Introduced or Extended

• Multinomial Probit
• Binary Probit
• Marginal Data Density
• Gibbs Sampler
• Metropolis-Hastings Algorithm

Entities Mentioned

• Siddhartha Chib
• Edward Greenberg

Quotes

"...the model is not commonly used, mainly because its likelihood function is difficult to evaluate except under simplifying assumptions... The purpose of this paper is to provide a unified simulation-based inference methodology for overcoming the problems in fitting multivariate probit models."

My Take

The canonical reference for Bayesian analysis of the multivariate probit model. The key methodological advance over Chib (1995) is handling the unknown normalising constant in the correlation prior by kernel density estimation — a technique that generalises the marginal likelihood identity to M-H settings (later formalised in Chib-Jeliazkov 2001). The GHK likelihood evaluation (eq. 11) is also used directly in Chib-Carlin (1999) Algorithm 7 for binary longitudinal models. The tailored independence/reflection chain proposal (eq. 7) is a practical workaround for correlated, bounded parameter spaces.