Petrone-Raftery (1997) A Note on the Dirichlet Process Prior

Summary

Petrone and Raftery consider Bayesian nonparametric inference for continuous-valued partially exchangeable data when the partition of observations into groups is unknown — a setting that includes change-point problems and mixture models. Using a mixture of products of Dirichlet processes as the prior, they show that the discreteness of the Dirichlet process can have a large effect on inference (posterior distributions and Bayes factors), sometimes producing conclusions quite different from those of a reasonable parametric model.

Key Claims

• Setting. Partially exchangeable continuous data with an unknown partition into groups (change-point, mixture, and more general hierarchical partition models); the partition is often the main object of inference.
• Prior. A mixture of products of Dirichlet processes (Cifarelli-Regazzini 1978), generalizing the Dirichlet process of Ferguson (1973, 1974).
• Discreteness matters. Because the DP is almost surely discrete (positive probability of ties), it distorts inference about the partition even for continuous data.
• Direction of the effect. When the observed data are all distinct, the DP prior's effect on the posterior is to favour more evenly balanced partitions, and its effect on Bayes factors is to favour more groups than a reasonable parametric model would.
• Reversal under hierarchy. In a hierarchical model with a Dirichlet process as the second-stage prior, the effect can be equally large but in the opposite direction — toward more unbalanced partitions.

Concepts Introduced or Extended

• Dirichlet Process Mixture
• Change-Point Analysis

Entities Mentioned

• Sonia Petrone
• Adrian E. Raftery

Quotes

"We show that the discreteness of the Dirichlet process can have a large effect on inference (posterior distributions and Bayes factors), leading to conclusions that can be different from those that result from a reasonable parametric model."

My Take

A sharp, useful caution to set against the enthusiasm of the DP-mixture literature: the very discreteness that makes the DP a natural clustering prior is a liability when the partition — how many groups, and which observations share one — is the inferential target, as in change-point and mixture problems. The finding that the prior systematically tilts Bayes factors toward more groups (and the hierarchical reversal toward fewer, more unbalanced ones) is exactly the kind of prior-sensitivity that is easy to miss when the DP is treated as an innocuous default. It pairs naturally with the discreteness and base-measure-sensitivity themes already flagged there, and is a reminder that "nonparametric" does not mean "assumption-free."