Tierney, Kass & Kadane (1989a) Approximate Marginal Densities of Nonlinear Functions

Summary

Tierney, Kass & Kadane (1989a) extend the Laplace approximation for marginal posterior densities from coordinate functions (Tierney-Kadane 1986) to arbitrary smooth nonlinear functions $g(\theta)$. For a joint density $p(\theta) \propto b(\theta)\exp\{-H(\theta)\}$ with a dominant mode at $\hat\theta$, the approximate marginal density of $g(\theta)$ at $y$ (eq. 3) requires only constrained minimisation of $H$ subject to $g(\theta)=y$ and evaluation of the Hessian at the constrained mode — no explicit global reparameterisation. The approximation achieves saddlepoint accuracy (relative errors $O(n^{-1})$) and is equivariant under invertible reparameterisations of $g$.

Key Claims

• Approximation (eq. 3): For $p(\theta) = cb(\theta)\exp\{-H(\theta)\}$ with dominant mode $\hat\theta$, the approximate marginal density of $g(\theta)$ at $y$ is: $$\tilde{p}(y) = (2\pi)^{k/2} \frac{[\det\Omega(\hat\theta(y))]^{1/2}}{[\det\Omega]^{1/2}[\det(Dg'\Omega(\hat\theta(y))Dg)]^{1/2}} b(\hat\theta(y))\exp\{-H(\hat\theta(y))\}$$ where $\Omega = [\nabla^2 H(\hat\theta)]^{-1}$, $\Omega(y) = [\nabla^2 H(\hat\theta(y))]^{-1}$, $\hat\theta(y)$ minimises $H$ s.t. $g(\theta)=y$, and $Dg$ is the Jacobian of $g$ evaluated at $\hat\theta(y)$.
• Saddlepoint accuracy: Relative errors are uniformly $O(n^{-1})$ on any fixed neighbourhood of $\hat g = g(\hat\theta)$. The renormalized approximation achieves $O(n^{-3/2})$ on $n^{-1/2}$-shrinking neighbourhoods — same as the TK (1986) coordinate-margin formula.
• Equivariance: If $f(\theta) = \alpha(g(\theta))$ for invertible smooth $\alpha$, then $\tilde p_f(t) = \tilde p_g(\alpha^{-1}(t))|\det D\alpha(\alpha^{-1}(t))|^{-1}$. (Not invariant to nonlinear reparameterisations of $\theta$ itself.)
• Derivation: Apply Laplace's method in the local coordinate system $(g(\theta), \varphi_2)$ near $\hat\theta$. The local transformation Jacobian cancels in the numerator/denominator ratio, yielding (3) without a global reparameterisation.
• Application 1 — marginal posterior: Two-year survival probability for Feigl-Zelen (1965) exponential survival model (17 AG-positive leukemia patients). Approximation and trapezoidal-rule exact density are visually indistinguishable.
• Application 2 — marginal sampling density: $\hat P = \Phi((x_0 - \bar X)/S)$ from a normal population; closed-form approximation derived and compared to exact noncentral-t result. Quality depends on parameterisation: $S > \log S > S^2$.
• Application 3 — saddlepoint marginalisation: Applied to the saddlepoint approximation to the joint density of the sufficient statistic $T_n$ in a multivariate exponential family; yields closed-form approximate marginal density of $g(T_n)$ for any smooth $g$.
• Related approximation (Hsu-Leonard-Tsui): Intermediate approximation (eq. 7) expands $\log p$ about $\hat\theta(y)$ to second order without requiring $g$ to be smooth — can handle non-differentiable $g$ by simulating from the resulting local normal.

Concepts Introduced or Extended

• Laplace Approximation

Entities Mentioned

• Luke Tierney
• Robert E. Kass
• Joseph B. Kadane

Quotes

"This paper presents an asymptotic approximation for the marginal density of a nonlinear function $g(\theta)$ that is applicable when the joint density of $\theta$ is dominated by a single mode and the Jacobian of $g$ is of full rank near that mode."

"Approximation (3) requires that the gradient of g be nonzero, or the Jacobian be of full rank, near the mode of the joint density p. Note that, as in (2), the normalizing constant c in (1) need not be evaluated since it cancels from the numerator and denominator in (3)."

My Take

This is the Bayesian computation equivalent of the delta method, but asymptotically superior: saddlepoint accuracy rather than first-order normality. The key advance over Tierney-Kadane (1986) is handling arbitrary smooth $g$ rather than only coordinate margins, enabling marginal inference for quantities of primary interest (e.g., a survival probability, a proportion, a nonlinear contrast) without explicit reparameterisation. The companion TKK (1989b) improves accuracy to $O(n^{-2})$ for posterior expectations via the fully exponential form. Both are of largely historical interest post-MCMC (Gelfand-Smith 1990), but the framework underlies the integrated nested Laplace approximation (INLA) and the Bayesian information criterion (BIC) approximation to marginal likelihood used throughout econometrics for lag selection and model comparison.