Change-Point Analysis

Definition

Change-point analysis concerns inference about the number and locations of structural breaks in a sequence of observations, where the data-generating process shifts at unknown times. In a piecewise-constant rate model for a Poisson process on $[0, L]$ , there are $k$ change-points $0 < s_1 < \cdots < s_k < L$ partitioning the timeline into $k+1$ segments with constant rates $h_0, h_1, \ldots, h_k$ respectively; both $k$ and the parameters $\{s_j, h_j\}$ are unknown. Bayesian change-point analysis places a prior over $k$ and estimates the joint posterior $p(k, s_1,\ldots,s_k, h_0,\ldots,h_k \mid y)$ via reversible-jump Markov chain Monte Carlo (RJMCMC).

Key Ideas

Unknown dimension: the number of change-points $k$ is itself unknown, making the problem trans-dimensional; standard fixed- $k$ MCMC cannot traverse different model sizes.
Hierarchical prior: $k \sim \text{Poisson}(\lambda)$ truncated at $k_{\max}$ ; conditional on $k$ , step positions $(s_1,\ldots,s_k)$ are the even-order statistics from $2k+1$ points uniformly distributed on $[0,L]$ (spacing prior that penalises very short intervals); step heights $h_j \sim \text{Gamma}(\alpha,\beta)$ independently.
Posterior inference: the target $p(k, s, h \mid y)$ is estimated by a single RJMCMC run; the fraction of samples at dimension $k$ estimates $p(k \mid y)$ directly; Bayes factors between $k$ values are their ratios adjusted for the prior.
Move types (Green 1995): height change H (perturb one $h_j$ at fixed $k$ ), position change P (relocate one $s_j$ at fixed $k$ ), birth (insert a new step, $k \to k+1$ ), death (remove a step, $k \to k-1$ ). Birth and death are dimension-changing moves handled by the RJMCMC bijection construction.
Spacing prior for step positions avoids short intervals that would be unpenalised by likelihood yet create spurious change-points — a defect of the independent uniform order-statistic prior.

How It Works

Green (1995) Poisson Process Model

For observations $y_1,\ldots,y_n$ from a Poisson process with piecewise-constant rate $x(t)$ on $[0,L]$ , the log-likelihood is

$\sum_i \log x(y_i) - \int_0^L x(t)\,dt$

Birth move ( $k \to k+1$ ): propose new step position $s^* \sim \text{Uniform}[0,L]$ ; $s^*$ falls in interval $(s_j, s_{j+1})$ with probability 1. The current height $h_j$ is split into $h_j'$ and $h_{j+1}'$ using $u \sim \text{Uniform}[0,1]$ :

$\frac{h_{j+1}'}{h_j'} = \frac{1-u}{u} \quad \text{preserving} \quad (s^* - s_j)\log h_j' + (s_{j+1} - s^*)\log h_{j+1}' = (s_{j+1} - s_j)\log h_j$

Acceptance probability: $\min\{1, R\}$ where $R = (\text{likelihood ratio}) \times (\text{prior ratio}) \times (\text{proposal ratio}) \times \text{Jacobian}$ , and the Jacobian is $(h_j' + h_{j+1}')^2/h_j$ .

Death move ( $k+1 \to k$ ): reverse of birth; select one step $s_{j+1}$ at random; merge $h_j$ and $h_{j+1}$ via the weighted geometric mean satisfying the same log-height conservation equation.

Coal Mining Disaster Application (Green 1995)

Data: $n=192$ coal mining disasters (serious accidents), 1851–1962, $L=40{,}907$ days. Hyperparameters: $\lambda=3$ , $k_{\max}=30$ , $\alpha=1$ , $\beta=200$ . Run: 40,000 iterations after 4,000 burn-in (45 seconds, Sun Sparc 2). Results: posterior mode $k=3$ change-points; main change-point near day 14420 ≈ 25 June 1890 (95% credible interval (CI): 24 May 1887–7 May 1896), consistent with Raftery-Akman (1986) and Carlin-Gelfand-Smith (1992) analyses. The posterior mean rate function shows a high accident rate 1851–1890 dropping sharply and plateauing around 0.002–0.003 events/day thereafter.

Wang-Zivot (2000) Multiple Structural Changes in AR Models

Wang and Zivot (2000) extend the Green (1995) framework to AutoRegressive (AR) time series with breaks in intercept, trend, and/or variance. For model (3) with $n$ breaks in the intercept: $u_t = \mu_1 I_{(1\le t<k_1)} + \mu_2 I_{(k_1\le t<k_2)} + \cdots + \mu_{n+1}I_{(k_n\le t<T)} + \rho u_{t-1} + \varepsilon_t$ conditional on $n$ , the break locations $k_j$ are given a uniform prior over all ordered subsequences. The conditional posterior of each $k_j$ given its neighbours $k_{j-1}$ and $k_{j+1}$ is multinomial, making Gibbs sampling straightforward. Wang and Zivot use the Bayesian Information Criterion (BIC) to select $n$ .

Metropolized Carlin-Chib (MCC) extension (Summers 2002): Summers (2002) treats $n$ itself as unknown by running independence reversible-jump MCMC. A flat prior is placed over $n \in \{0,\ldots,5\}$ ; a proposed $n_1$ is drawn, parameters for the $n_1$ -break model drawn from their full conditional posteriors (serving as proposal density), and the move accepted with probability equal to the standard Metropolis ratio. This is equivalent to Green's (1995) reversible-jump algorithm for the special case where the proposal is the full conditional — an "independence" variant that avoids specifying bijections between different-dimensional parameter spaces. Posterior probabilities of each $n$ are estimated as visit frequencies over 10,000 iterations.

Why It Matters

Bayesian change-point analysis provides a full posterior distribution over k rather than committing to a single model selection decision; model averaging is implicit.
RJMCMC (Green 1995) enables genuinely continuous-time treatment of change-point locations, avoiding discretisation artefacts of indicator-variable approaches (Carlin-Gelfand-Smith 1992, Barry-Hartigan 1993).
The framework extends naturally to two-dimensional change-point problems (image segmentation via Voronoi tessellation) and partition models.
The same coal mining data set is analysed by Raftery-Akman (1986) with one change-point, Carlin-Gelfand-Smith (1992) with discrete breaks, and Barry-Hartigan (1992, 1993) with product-partition models — Green (1995) provides the most general continuous-time treatment.

Open Questions

Choosing move probabilities $b_k$ and $d_k$ to balance exploration of different model sizes without excessive time at extremes.
Convergence assessment for trans-dimensional chains — standard diagnostics (Gelman-Rubin $\hat{R}$ ) apply only within each $k$ subspace.
Non-Poisson arrival processes and regression change-point models (unknown break in regression coefficients) require different bijection constructions.