Zellner-Tobias-Ryu (1999) Bayesian Method of Moments Analysis of Parametric and Semiparametric Regression Models

Summary

Extends the Bayesian Method of Moments (BMOM) framework to semiparametric regression, where the regression function $f(x)$ is unknown and approximated by a series expansion (Polynomial, Fourier, or Gallant flexible functional form). The key BMOM idea is to construct a post-data density for parameters by maximizing entropy subject to sample moment constraints, entirely avoiding specification of a likelihood function. Three model-selection tools are developed for choosing the expansion type and number of terms: estimation loss, predictive loss, and BMOM posterior odds. A Monte Carlo experiment with data generated from a constant elasticity of substitution (CES) production function illustrates the methods.

Key Claims

• BMOM yields finite-sample post-data densities and predictive densities for parameters without assuming a likelihood, resolving what Zellner calls the "inverse problem" posed by Bayes (1763) without imposing potentially false distributional assumptions.
• For a parametric regression $y = W\delta + u$ with moment constraints $E[\delta|D] = \hat\delta$ (ordinary least squares, OLS) and $\text{Cov}[\delta|D] = s^2(W'W)^{-1}$, the MaxEnt posterior is multivariate Normal — identical to the diffuse-prior Bayesian posterior, but derived without any likelihood specification.
• For semiparametric regression, a series expansion $y = \sum_j \beta_j\psi_j(x) + \varepsilon$ reduces the problem to a parametric BMOM analysis; the choice of basis functions $\psi_j(x)$ (Polynomial, Fourier, Gallant) and number of terms $J$ is treated as a model-selection problem.
• Estimation loss criterion: choose $J$ to minimise trace of the posterior covariance of $\hat\delta$, i.e., the sum of posterior variances of the regression coefficients.
• Predictive loss criterion: ratio $R = [m + a_1]s_1 / [m + a_2]s_2$ compares expected squared prediction error for $m$ future observations across two expansions; select whichever achieves $R < 1$.
• Posterior odds criterion: when both models have Normal predictive densities $N(\hat y_i, V_i)$ for future observations $y_f$, $\log K = \log(P_1/P_2) - (1/2)\log|V_1/V_2| - (1/2)[(y_f - \hat y_1)'V_1^{-1}(y_f - \hat y_1) - (y_f - \hat y_2)'V_2^{-1}(y_f - \hat y_2)]$; related to the Jeffreys–Kullback-Leibler (KL) divergence measure $W = \text{CE}(f_1, f_2) + \text{CE}(f_2, f_1)$, where CE denotes cross-entropy.
• Monte Carlo CES production function experiment: posterior odds strongly favour parsimonious Polynomial and Fourier expansions (2–4 terms) over Gallant series; base-model posterior odds ratios on the order of $10^{-11}$ against Gallant when sample size is large.

Concepts Introduced or Extended

• Bayesian Method of Moments
• Bayesian Linear Regression

Entities Mentioned

• Arnold Zellner

Quotes

"One of the distinguishing features of the BMOM approach is that it yields post-data densities for models' parameters without use of an assumed likelihood function."

"BMOM gives good answers for the questions it addresses while not purporting to go beyond the information that is really there in the prior and the data." — Laskey (1997), commenting on BMOM

My Take

This paper is primarily a methodological demonstration rather than an econometric application paper. The core BMOM result for regression (MaxEnt posterior = diffuse-prior Bayesian posterior) is elegant but also somewhat deflating: you end up at the same place as standard Bayesian regression, just by a different path. The value is clearest when the likelihood is genuinely unknown — e.g. in semiparametric settings or when error distributions are highly uncertain. The three model-selection tools (estimation loss, predictive loss, posterior odds) are well-motivated and practically useful for series-expansion truncation. The Monte Carlo results are somewhat limited — a single data generating process (DGP; CES) under a single sample size scenario — but illustrate the approach clearly. Connection to the KL divergence in posterior odds is the most durable theoretical insight.