Albert-Chib (1997b) Bayesian Methods for Cumulative, Sequential and Two-Step Ordinal Data Regression Models

Summary

Albert and Chib (1997b) develop a unified Bayesian Markov chain Monte Carlo (MCMC) framework for fitting, criticizing, and comparing three non-nested ordinal regression models — the cumulative probit (McCullagh 1980), the sequential probit (Tutz 1990, 1991), and the two-step compound model (Tutz 1989). Because the models are non-nested, classical likelihood ratio tests are inapplicable; the paper instead uses Bayes factors (BFs) computed from Chib (1995) marginal likelihoods. The central algorithmic innovation is a log-spacing reparameterization of the ordered cut-points in the cumulative model that lifts them to an unrestricted real vector, enabling a tuned multivariate-$t$ Metropolis-Hastings (MH) step that mixes "extremely well." Three methods for eliciting proper priors (needed for valid Bayes factors) are developed: training sample, imaginary prior sample, and Dirichlet-on-multinomial.

Key Claims

• Three non-nested ordinal models: cumulative model $\Pr(Y \leq j \mid \beta, \gamma) = \Phi(\gamma_j - x_i'\beta)$ with ordered cut-points; sequential model $\Pr(Y_i = j \mid Y_i \geq j, \beta, \gamma) = \Phi(\gamma_j - x_i'\beta)$ with unrestricted thresholds (discrete-hazard / continuation-ratio structure); two-step (compound) model partitioning $J+1$ categories into $s$ subsets with a cumulative model at subset level and separate cumulative models within each subset.
• Cut-point reparameterization (Algorithm 1): transform ordered $\gamma_1 \leq \cdots \leq \gamma_{J-1}$ to unrestricted $\alpha$ via $\alpha_1 = \log \gamma_1$, $\alpha_j = \log(\gamma_j - \gamma_{j-1})$ for $j \geq 2$; inverse $\gamma_j = \sum_{i=1}^j \exp(\alpha_i)$; MH proposal is a multivariate-$t$ with mode $\hat\alpha$ and curvature $V$ from Newton-Raphson on $\ln \pi(\alpha \mid y, \beta)$.
• Sequential model latent structure: each observation $Y_i = j$ requires $j_i = \min\{j+1, J\}$ category-specific independent latent variables $z_{ij} \sim N(x_i'\beta, 1)$, giving a variable-dimension augmented data vector.
• Two-step posterior factorization: the joint posterior factors into $t+1$ independent components — subset-level ($\{\gamma_j\}, \beta_0$) and within-subset ($\{\theta_{j,r}\}, \beta_j$) for $j=1,\ldots,t$ — so Algorithm 3 is simply $t+1$ independent runs of Algorithm 1.
• Marginal likelihood via Chib (1995): identity $m(y) = f(y \mid \psi^*)\pi(\psi^*)/\pi(\psi^* \mid y)$ with posterior ordinate decomposed as $\pi(\beta^*, \alpha^* \mid y) = \pi(\beta^* \mid y)\pi(\alpha^* \mid y, \beta^*)$; $\beta^*$ ordinate estimated by averaging $\phi_k(\beta^* \mid \hat\beta^{(g)}, B^{-1})$ over MCMC $z$-draws; $\alpha^*$ ordinate estimated by Gaussian kernel smoothing (Scott 1992 bandwidth) from a reduced run fixing $\beta = \beta^*$.
• Three prior elicitation methods: (a) training sample — split data, fit flat-prior model to training portion, use posterior moments as hyperparameters; (b) imaginary prior sample — artificial dataset with equal ordinal level across all cells and no covariate associations; (c) Dirichlet-on-multinomial — Dirichlet distributions on response probability vectors $\pi_k$ at chosen covariate values, transformed via Jacobian to a prior on $(\beta, \gamma)$.
• Bayesian latent residuals: $r_i = z_i - x_i'\beta$ with prior $r_i \sim N(0,1)$; tail probability $\Pr(|r_i| > 2 \mid y) > 0.05$ flags poorly fitted observations; probability plot of posterior-mean ordered residuals against $N(0,1)$ order statistics detects global misfit.
• National Longitudinal Survey of Youth (NLSY) educational attainment application ($n=500$ subsample, 5 categories, training sample prior): cumulative model wins over sequential ($\log \mathrm{BF} = 3.15$); parent education dominant ($\log_{10} \mathrm{BF} \approx 18.96$); race and region should be excluded ($\log_{10} \mathrm{BF} \approx -1.2$ to $-2.0$); gender marginal ($\log_{10} \mathrm{BF} \approx 0.02$).
• Canada General Social Survey (GSS) physician visits application ($n=1{,}000$, two-step with 4 subsets): full two-step beats collapsed 4-category model ($\log \mathrm{BF}_{21} = -10.7$); model with $\beta_2 = 0$ slightly preferred over full model ($\log \mathrm{BF}_{31} = 1.4$), indicating covariates do not explain variation among high-visit counts.
• Algorithmic performance: 5,000 iterations / 100 burn-in; $\tau = 1$, $\nu = 30$ tuning constants; Algorithm 1 trace plots and autocorrelations show rapid mixing; results insensitive to MCMC design parameters.

Concepts Introduced or Extended

• Ordinal Regression
• Sequential Ordinal Model
• Binary Probit
• Bayesian Residual Analysis
• Marginal Data Density

Entities Mentioned

• James H. Albert
• Siddhartha Chib

Quotes

"The feature that makes this problem interesting is that the models are non-nested and standard classical methods for comparing these models are either inappropriate or difficult to apply." (p. 2)

"Our empirical experience with Algorithm 1 is that it is fast and mixes extremely well." (p. 11)

My Take

The paper's key technical contribution — the log-spacing reparameterization of cut-points — is elegant and practically important: it converts a constrained MH problem into an unconstrained one at the cost of a Jacobian, avoiding the ordered-parameter complications of Cowles (1996) and Chen-Nandram (1996). The three prior elicitation methods address a real gap: Bayes factors are sensitive to prior specification for non-nested models, and none of the three methods is obviously dominant (training sample wastes data; imaginary sample is artificial; Dirichlet-on-multinomial requires specifying covariate scenarios). The published follow-up, Albert-Chib (2001), focuses exclusively on the sequential model with the hospital length-of-stay application; the cumulative and two-step results here appear to remain as working paper contributions. The educational attainment Bayes factors (parent education $\log_{10} \mathrm{BF} \approx 19$) are striking but depend heavily on the training-sample prior — a sensitivity analysis would strengthen the conclusions.