Bridge Sampling

Definition

Bridge sampling (Meng-Wong 1996) is a Monte Carlo method for estimating a ratio of normalizing constants — and hence a Bayes factor or a single marginal likelihood — by connecting two densities through an intermediate "bridge" function. For two densities known only up to their constants, $p_i(\theta)=q_i(\theta)/c_i$ ($i=1,2$), the identity $$\frac{c_1}{c_2} = \frac{\mathrm{E}_{2}[\,q_1(\theta)\,\alpha(\theta)\,]}{\mathrm{E}_{1}[\,q_2(\theta)\,\alpha(\theta)\,]}$$ holds for any bridge function $\alpha(\theta)$ with finite, nonzero expectations, where $\mathrm{E}_i$ denotes expectation under $p_i$. Both expectations are estimated from draws already available under each density, so the constant ratio is recovered without sampling from any bridging path.

Key Ideas

• A ratio estimator, not a single-density estimator. Bridge sampling natively estimates $c_1/c_2$. To obtain one marginal likelihood $c_1=p(y)$, take $p_2$ to be a normalized proposal (e.g. a moment-matched Gaussian on the posterior) whose constant is known.
• Generalises importance and harmonic-mean estimators. Choosing $\alpha=1/q_2$ recovers ordinary importance sampling; $\alpha=1/(q_1 q_2)$ recovers the notoriously unstable harmonic-mean estimator. Bridge sampling's freedom in $\alpha$ is exactly what lets it avoid their failure modes.
• Optimal bridge and overlap. The variance-minimising choice is $\alpha \propto 1/\big(s_1 p_1 + s_2 p_2\big)$ (with $s_i$ the sampling fractions), solved by a short fixed-point iteration. Accuracy hinges on the overlap between $p_1$ and $p_2$: little overlap means high variance, which is why a well-tuned proposal matters.
• Path/thermodynamic sampling as the limit. When the two densities barely overlap, one inserts a sequence of intermediate densities; the continuous limit is path sampling (thermodynamic integration), a close relative that trades more draws for robustness across widely separated densities.
• More stable than harmonic mean, cheaper than adding a sampler. It reuses posterior draws, so it is a common upgrade path from the harmonic-mean or naive importance-sampling marginal-likelihood estimates that are known to be unreliable.

How It Works

1. Obtain draws from the posterior $p_1(\theta)=p(\theta\mid y)$ (unnormalized $q_1=p(y\mid\theta)p(\theta)$, target constant $c_1=p(y)$) and construct a normalized proposal $p_2$ with known constant, drawing a sample from it.
2. Initialise the constant ratio (e.g. by importance sampling).
3. Iterate the fixed-point recursion for the optimal bridge, alternately updating $\alpha$ and the current estimate of $c_1/c_2$ from the two samples, until convergence.
4. Return $\hat p(y)=c_1$; for a Bayes factor between two models, either form the ratio of two such marginal likelihoods or bridge the two posteriors directly.

Why It Matters

• Robust marginal likelihoods for model comparison. Where the harmonic-mean estimator has infinite variance and plain importance sampling degrades in high dimensions, bridge sampling gives stable $p(y)$ estimates — the quantity that drives Bayes factors, Bayesian model selection, and Bayesian model averaging weights.
• A benchmark method in the toolkit. Comparative reviews of Bayes-factor computation (e.g. Han-Carlin (2001)) situate bridge/path sampling alongside the Chib (1995) marginal-likelihood identity and reversible-jump MCMC as the reliable options for standard hierarchical models.
• Reuses existing output. Because it works from draws an analyst already has, bridge sampling adds model-comparison capability without designing a new trans-dimensional sampler.

Open Questions

• Proposal construction in high dimensions. Building a $p_2$ with adequate posterior overlap is hard when the posterior is high-dimensional or multimodal; warp transformations and mixture proposals help but need tuning.
• Bridge vs. Chib vs. reversible jump. Which estimator is most efficient is problem-dependent — Chib's identity excels for Gibbs-sampled hierarchical models, reversible jump for variable-dimension spaces, bridge/path for smooth continuous parameter spaces.
• Error assessment. Reliable Monte Carlo standard errors for the fixed-point estimator, especially under poor overlap, remain an area of active methodology.

Related

• Marginal Data Density
• Bayes Factor
• Importance Sampling
• Chib (1995) Marginal Likelihood from the Gibbs Output
• Model Comparison
• Bayesian Model Selection
• Bayesian Model Averaging
• Han-Carlin (2001) MCMC Methods for Computing Bayes Factors: A Comparative Review
• Siddhartha Chib