Sethuraman (1994) A Constructive Definition of Dirichlet Priors

Summary

Sethuraman gives the stick-breaking construction of the Dirichlet process: an explicit, elementary recipe that builds a random probability measure $P$ as a countable weighted sum of point masses, and proves it satisfies the three defining properties of a Dirichlet measure. The construction works on arbitrary measurable spaces (no separable-complete-metric-space restriction), uses unordered weights (simpler than Ferguson's decreasing-order weights), and turns the theory of Dirichlet priors — conjugacy, discreteness, posterior form — into short self-contained proofs via a single self-similar distributional equation. It is the representation on which essentially all modern Bayesian-nonparametric computation is built.

Key Claims

• The construction. With base measure $\alpha=M\beta$ ($\beta=\alpha/\alpha(\mathcal X)$ the normalized base, $M=\alpha(\mathcal X)$ the total mass), draw atoms $Y_1,Y_2,\dots \overset{iid}{\sim}\beta$ and "failure rates" $\theta_1,\theta_2,\dots \overset{iid}{\sim}\text{Beta}(1,M)$, independent of the $Y$'s. Set the stick-breaking weights $p_1=\theta_1$, $p_n=\theta_n\prod_{m<n}(1-\theta_m)$ (which sum to 1 a.s.), and define $$P=\sum_{n=1}^{\infty} p_n\,\delta_{Y_n}. \tag{2.1}$$ Then $P\sim \mathcal D_\alpha$, the Dirichlet process with parameter $\alpha$.
• Three properties, cleanly proved. P1: $\mathcal D_\alpha$ is a probability measure on the space of probability measures; P2: it puts probability one on the set of discrete measures (immediate from the construction); P3: the posterior after observing $X$ is again Dirichlet, $\mathcal D_{\alpha+\delta_X}$ (conjugacy).
• Self-similar distributional equation. $P \overset{d}{=} \theta_1\delta_{Y_1} + (1-\theta_1)P'$, with $P'$ an independent copy of $P$. This one fixed-point equation (unique solution) drives the whole proof that $P\sim\mathcal D_\alpha$ and is the structural heart of the representation.
• No topological restrictions. Unlike Ferguson (1973) (Kolmogorov consistency / Harris lemma, needing a nice space) and Blackwell-MacQueen (1973) (de Finetti + generalized Pólya urn, needing a separable complete metric space), the construction is a plain product-measure argument valid for any $(\mathcal X,\mathcal B)$.
• Unordered vs. ordered weights. Ferguson's Section-4 construction uses weights sorted in decreasing order; Sethuraman's unordered weights are the same set rearranged but yield a far simpler algebra (and there is no easy way to un-order Ferguson's).

Concepts Introduced or Extended

• Dirichlet Process Mixture

Entities Mentioned

• Jayaram Sethuraman
• Thomas S. Ferguson
• Ferguson (1973) A Bayesian Analysis of Some Nonparametric Problems

Quotes

"In this paper we give a simple and new constructive definition of Dirichlet measures removing the restriction that the basic space should be $R_k$."

"The use of unordered weights in this paper simplifies all our calculations."

My Take

This is one of those papers whose representation became more important than its stated theorems: Ferguson (1973) had already defined the Dirichlet process and proved conjugacy, but Sethuraman's $P=\sum p_n\delta_{Y_n}$ with $\text{Beta}(1,M)$ stick-breaks is what made the DP computable — truncated/blocked Gibbs samplers (Ishwaran-James), slice samplers, variational methods, and every hierarchical/dependent-DP generalization take this as their starting point. The self-similar equation $P\overset{d}{=}\theta_1\delta_{Y_1}+(1-\theta_1)P'$ is the whole idea in one line: break off a $\text{Beta}(1,M)$ fraction of the remaining stick, drop an atom there, recurse. For the wiki it is the foundational source behind the stick-breaking bullet in Dirichlet Process Mixture — the constructive counterpart to Antoniak's (1974) cluster-count law — and the direct enabler of the DP-mixture samplers that Neal (2000) systematizes.