Change-Point Analysis

Definition

Change-point analysis concerns inference about the number and locations of structural breaks in a sequence of observations, where the data-generating process shifts at unknown times. In a piecewise-constant rate model for a Poisson process on $[0, L]$ , there are $k$ change-points $0 < s_1 < \cdots < s_k < L$ partitioning the timeline into $k+1$ segments with constant rates $h_0, h_1, \ldots, h_k$ respectively; both $k$ and the parameters $\{s_j, h_j\}$ are unknown. Bayesian change-point analysis places a prior over $k$ and estimates the joint posterior $p(k, s_1,\ldots,s_k, h_0,\ldots,h_k \mid y)$ via reversible-jump Markov chain Monte Carlo (RJMCMC).

Key Ideas

Unknown dimension: the number of change-points $k$ is itself unknown, making the problem trans-dimensional; standard fixed- $k$ MCMC cannot traverse different model sizes.
Hierarchical prior: $k \sim \text{Poisson}(\lambda)$ truncated at $k_{\max}$ ; conditional on $k$ , step positions $(s_1,\ldots,s_k)$ are the even-order statistics from $2k+1$ points uniformly distributed on $[0,L]$ (spacing prior that penalises very short intervals); step heights $h_j \sim \text{Gamma}(\alpha,\beta)$ independently.
Posterior inference: the target $p(k, s, h \mid y)$ is estimated by a single RJMCMC run; the fraction of samples at dimension $k$ estimates $p(k \mid y)$ directly; Bayes factors between $k$ values are their ratios adjusted for the prior.
Move types (Green 1995): height change H (perturb one $h_j$ at fixed $k$ ), position change P (relocate one $s_j$ at fixed $k$ ), birth (insert a new step, $k \to k+1$ ), death (remove a step, $k \to k-1$ ). Birth and death are dimension-changing moves handled by the RJMCMC bijection construction.
Spacing prior for step positions avoids short intervals that would be unpenalised by likelihood yet create spurious change-points — a defect of the independent uniform order-statistic prior.
Fixed- $k$ Gibbs sampling (Carlin-Gelfand-Smith 1992): the earlier, non-trans-dimensional route. Conditional on a known number of changepoints, a hierarchical Bayes model is fit by the Gibbs sampler — cycling through the full conditionals of the changepoint locations and the segment parameters — which sidesteps the high-dimensional analytic/numerical integration that changepoint problems had previously required. The same generic engine handles changing regressions, changing Poisson processes, and changing Markov chains, making previously intractable models routine; the unknown- $k$ case is then handled on top by model comparison or the RJMCMC birth/death moves above.

How It Works

Green (1995) Poisson Process Model

For observations $y_1,\ldots,y_n$ from a Poisson process with piecewise-constant rate $x(t)$ on $[0,L]$ , the log-likelihood is

$\sum_i \log x(y_i) - \int_0^L x(t)\,dt$

Birth move ( $k \to k+1$ ): propose new step position $s^* \sim \text{Uniform}[0,L]$ ; $s^*$ falls in interval $(s_j, s_{j+1})$ with probability 1. The current height $h_j$ is split into $h_j'$ and $h_{j+1}'$ using $u \sim \text{Uniform}[0,1]$ :

$\frac{h_{j+1}'}{h_j'} = \frac{1-u}{u} \quad \text{preserving} \quad (s^* - s_j)\log h_j' + (s_{j+1} - s^*)\log h_{j+1}' = (s_{j+1} - s_j)\log h_j$

Acceptance probability: $\min\{1, R\}$ where $R = (\text{likelihood ratio}) \times (\text{prior ratio}) \times (\text{proposal ratio}) \times \text{Jacobian}$ , and the Jacobian is $(h_j' + h_{j+1}')^2/h_j$ .

Death move ( $k+1 \to k$ ): reverse of birth; select one step $s_{j+1}$ at random; merge $h_j$ and $h_{j+1}$ via the weighted geometric mean satisfying the same log-height conservation equation.

Coal Mining Disaster Application (Green 1995)

Data: $n=192$ coal mining disasters (serious accidents), 1851–1962, $L=40{,}907$ days. Hyperparameters: $\lambda=3$ , $k_{\max}=30$ , $\alpha=1$ , $\beta=200$ . Run: 40,000 iterations after 4,000 burn-in (45 seconds, Sun Sparc 2). Results: posterior mode $k=3$ change-points; main change-point near day 14420 ≈ 25 June 1890 (95% credible interval (CI): 24 May 1887–7 May 1896), consistent with Raftery-Akman (1986) and Carlin-Gelfand-Smith (1992) analyses. The posterior mean rate function shows a high accident rate 1851–1890 dropping sharply and plateauing around 0.002–0.003 events/day thereafter.

Wang-Zivot (2000) Multiple Structural Changes in AR Models

Wang and Zivot (2000) extend the Green (1995) framework to AutoRegressive (AR) time series with breaks in intercept, trend, and/or variance. For model (3) with $n$ breaks in the intercept: $u_t = \mu_1 I_{(1\le t<k_1)} + \mu_2 I_{(k_1\le t<k_2)} + \cdots + \mu_{n+1}I_{(k_n\le t<T)} + \rho u_{t-1} + \varepsilon_t$ conditional on $n$ , the break locations $k_j$ are given a uniform prior over all ordered subsequences. The conditional posterior of each $k_j$ given its neighbours $k_{j-1}$ and $k_{j+1}$ is multinomial, making Gibbs sampling straightforward. Wang and Zivot use the Bayesian Information Criterion (BIC) to select $n$ .

Product Partition Model (Barry-Hartigan 1993)

An alternative to reversible-jump that avoids trans-dimensional moves entirely. The timeline is partitioned into contiguous blocks of constant parameter value (block starts = change points), and the prior probability of a partition $p$ is proportional to a product of prior cohesions, $P(p)\propto\prod_{\text{blocks }ij}c_{ij}$ , with block parameters independent given the partition (Barry-Hartigan 1993). The decisive property is closure: multiplying this product-form prior by a product-form likelihood leaves the posterior a product partition model, with posterior cohesions $c_{ij}f_{ij}(X_{ij})$ (prior cohesion × block marginal likelihood) and updated independent block posteriors. A common cohesion makes each position a change point with probability $p$ independently (equivalently a Markov parameter sequence that stays constant w.p. $1-p$ ); for normal data the block mean has prior $\mathcal N(\mu_0,\sigma_0^2/(j-i))$ , so short blocks are allowed larger deviations. Parameter estimates average over all partitions (a Bayesian model average, not a single chosen segmentation) and are computable exactly in $O(n^2)$ or by $O(n)$ -per-sweep Gibbs sampling over the change-point indicators. Barry-Hartigan report it avoids the failure modes of Schwarz-BIC, Chernoff-Zacks, and Yao estimators and is superior for sharp, short-lived changes.

Metropolized Carlin-Chib (MCC) extension (Summers 2002): Summers (2002) treats $n$ itself as unknown by running independence reversible-jump MCMC. A flat prior is placed over $n \in \{0,\ldots,5\}$ ; a proposed $n_1$ is drawn, parameters for the $n_1$ -break model drawn from their full conditional posteriors (serving as proposal density), and the move accepted with probability equal to the standard Metropolis ratio. This is equivalent to Green's (1995) reversible-jump algorithm for the special case where the proposal is the full conditional — an "independence" variant that avoids specifying bijections between different-dimensional parameter spaces. Posterior probabilities of each $n$ are estimated as visit frequencies over 10,000 iterations.

Change-points as hidden-Markov durations (Chib 1998; Koop-Potter 2004)

For time-series change-point problems a different construction dominates. Chib (1998) attaches a latent regime-indicator Markov chain to the observations and converts "where are the breaks?" into "how long does each regime last?" — the change-points are the durations of the Markov regimes, sampled by Chib's (1996) forward-filter/backward-sample algorithm (valid even for non-time-homogeneous chains). Because within-regime exit probability is constant, this construction forces a Geometric duration distribution: a regime is most likely to end in its first period and monotonically less likely thereafter.

Koop and Potter (2004) relax this in two ways: (i) they let the number of change-points be unknown and unbounded, avoiding the awkward prior a fixed count imposes (fixing $K$ breaks forces a non-time-homogeneous chain at the sample end with undesirable prior properties); and (ii) they replace the Geometric duration with a Poisson duration inside a hierarchical prior, giving a non-monotone, more realistic regime-length distribution while keeping conditionally-conjugate updates. The resulting model approximately nests both the time-varying-parameter model (a break every period) and the small-fixed- $K$ change-point model, and — applied to US GDP and PCE inflation — finds more breaks than earlier work, behaving like a TVP-with-stochastic-volatility model with heterogeneous transition innovations. The distinction matters most for forecasting, where only a model admitting future breaks of history-dependent size yields honest predictive uncertainty.

The companion Koop-Potter (2006) adds a full hierarchical prior on both durations and regime parameters and shows how to do real-time forecasting with the model by writing it in state-space form and running a sequential importance sampler: as each new observation arrives the importance weights are updated recursively — $w_{T+1}\propto p(y_{T+1}\mid z_{T+1})p(z_{T+1}\mid z_T)/q(\cdot)$ — so predictive densities roll forward without re-running the MCMC (a Negative-Binomial regime-count tail supplies extra future-regime durations when the horizon needs them).

Why It Matters

Bayesian change-point analysis provides a full posterior distribution over k rather than committing to a single model selection decision; model averaging is implicit.
RJMCMC (Green 1995) enables genuinely continuous-time treatment of change-point locations, avoiding discretisation artefacts of indicator-variable approaches (Carlin-Gelfand-Smith 1992, Barry-Hartigan 1993).
The framework extends naturally to two-dimensional change-point problems (image segmentation via Voronoi tessellation) and partition models.
The same coal mining data set is analysed by Raftery-Akman (1986) with one change-point (conjugate Gamma-Poisson posterior + a Bayes-factor test for whether a change exists at all, using the Spiegelhalter-Smith imaginary-observation device to make the test well-defined under vague priors), Carlin-Gelfand-Smith (1992) with discrete breaks, and Barry-Hartigan (1992, 1993) with product-partition models — Green (1995) provides the most general continuous-time treatment.

Open Questions

Choosing move probabilities $b_k$ and $d_k$ to balance exploration of different model sizes without excessive time at extremes.
Convergence assessment for trans-dimensional chains — standard diagnostics (Gelman-Rubin $\hat{R}$ ) apply only within each $k$ subspace.
Non-Poisson arrival processes and regression change-point models (unknown break in regression coefficients) require different bijection constructions.