Geweke-Keane (2007) Smoothly Mixing Regressions

Summary

Geweke and Keane propose the smoothly mixing regression (SMR) model for full Bayesian inference about the conditional distribution $p(y|x)$, not merely the conditional mean. The key extension over a standard mixture of normals is that state probabilities depend on observed covariates through a multinomial probit model, making the mixing weights continuous functions of $x$. Markov chain Monte Carlo (MCMC) estimation uses a five-block Gibbs sampler with a Metropolis-within-Gibbs step for the latent weight matrix. The framework provides a Bayesian alternative to frequentist quantile regression that yields the full conditional distribution, is monotonicity-consistent, and outperforms t-GARCH(1,1) in out-of-sample S&P 500 forecasting.

Key Claims

• State-dependent mixing: Standard finite mixture has state probabilities $\pi_j$ fixed; SMR replaces these with $P(\tilde s_t = j | z_t)$ depending on covariates $z_t$ through a multinomial probit on latent $\tilde w_t \sim \mathcal{N}(\Gamma z_t, I_m)$, $\tilde s_t = j$ iff $\tilde w_{tj} \geq \tilde w_{ti}$ for all $i$.
• Full model (eq. 1, 4–5): $y_t | (u_t, v_t, \tilde s_t = j) \sim \mathcal{N}(\beta'u_t + \alpha_j'v_t,\; \sigma^2 \sigma_j^2)$ where $u_t$ governs common location, $v_t$ state-specific slopes, $z_t$ state probabilities. Five model configurations (A–E) vary which of $(u_t, v_t, z_t)$ are active.
• Why multinomial probit: Full multinomial probit (MNP) on $\tilde s_t$ with unconstrained variance $\Sigma$ produces multimodal likelihoods (Keane 1992); multinomial logit has nonconjugate conditionals with worse MCMC mixing. The $I_m$ variance identification is the cleanest solution.
• Single-dimension state probability integrals: $P(\tilde s_t = j | \Gamma, z_t) = \int_{-\infty}^{\infty} \phi(y - \gamma_j'z_t) \prod_{i \neq j} \Phi(y - \gamma_i'z_t)\, dy$ — only one quadrature dimension regardless of $m$, avoiding Geweke-Hajivassiliou-Keane (GHK) simulation.
• Five-block Gibbs sampler: Blocks $(\beta, A)$, $\sigma^2$, $\sigma_j^2$, $\Gamma^*$, $\tilde W$. The $\tilde W$ block uses Metropolis-within-Gibbs: draw $\tilde w_t^{\text{prop}} \sim \mathcal{N}(\Gamma z_t, I_m)$; accept with probability $\min\{\sigma_{j^*}^{-1} \exp[-(y_t - \alpha_{j^*}'v_t - \beta'u_t)^2/(2\sigma^2\sigma_{j^*}^2)] / \sigma_j^{-1} \exp[\cdots], 1\}$ where $j^*$ is the state implied by the proposal.
• Identification: Label switching handled by noting conditional distribution is invariant to state permutation; over-translation in $\Gamma$ resolved by restriction $\iota_m'\Gamma = 0$, reparameterized as $\Gamma^* = P_2'\Gamma$ for orthonormal $P_2$; over-scale by fixing $\text{Var}(\tilde w_t) = I_m$.
• Priors: Polynomial basis functions for $u_t, v_t, z_t$; grid-based Gaussian priors $C\beta | (\mu, \tau_\beta^2) \sim \mathcal{N}(\iota_r\mu, \tau_\beta^2 r I_r)$ — each grid point gets weight of roughly one observation. Independent inverted-gamma priors for $\sigma_j^2$ centered at 1 (identification via $\nu^*$).
• Model selection: Modified cross-validated log scoring rule (random hold-out subset) for cross-section; sequential predictive likelihoods $\sum_{s=T_1+1}^T \log p(y_s | Y_{s-1}, \text{SMR})$ for time series.
• Earnings application: 2,698 white Panel Study of Income Dynamics (PSID) men in 1993; $y = \log(\text{earnings})$, covariates age and education with polynomial basis. Best model: E with $m=3$, $L_a=4$, $L_b=2$. Quantiles are nearly always increasing in education; age profiles vary substantially by quantile; dispersion minimal and roughly constant for ages 25–55 with 10–15 years of education.
• S&P 500 application: 1990–1999 daily returns; covariates $a_t = y_{t-1}$ and $b_t = (1-g)\sum_{s=0}^\infty g^s |y_{t-2-s}|^\kappa$ (exponentially-weighted volatility). Best model: C ($m=3$, smooth mixture of normals). SMR predictive log-likelihood −1602.0 vs. t-GARCH(1,1) −1659.2 — decisive out-of-sample superiority; conditional mean roughly constant; leverage effect (greater volatility following large negative returns); returns approximately symmetric.

Concepts Introduced or Extended

• Smoothly Mixing Regressions
• Quantile Regression
• Mixture of Normals
• Multinomial Probit
• Markov Chain Monte Carlo

Entities Mentioned

• John Geweke
• Michael Keane

Quotes

"The smoothly mixing regression model is the combination of (1), (4) and (5). The adverb 'smoothly' derives from the fact that the conditional state probabilities implied by (4)-(5) are continuous functions of the covariates $Z = [z_1, \ldots, z_T]'$."

"Since we have presented only quantiles, perhaps it bears emphasis that our method allows access to the entire distribution in distinction to the methods surveyed in the introduction that provide only estimates of quantiles."

My Take

The SMR model is elegant: it recombines existing pieces (mixture of normals, multinomial probit data augmentation, Gibbs sampling) in a way that makes the full conditional distribution $p(y|x)$ tractable and flexible. The key insight that state probability integrals reduce to one-dimensional quadrature (regardless of $m$) makes the framework computationally competitive. The out-of-sample S&P 500 result is striking — a 57-point log-likelihood gap over t-GARCH(1,1) is very large. The main limitation is that the number of components $m$ is fixed and chosen by cross-validation rather than inferred; extending to a Dirichlet process prior is the natural next step the authors acknowledge but do not pursue.

Circulated as a 2005 working paper; published as Geweke, J. and M. Keane (2007), "Smoothly Mixing Regressions," Journal of Econometrics 138(1): 252–290 (the citation of record).