Ben-Hur, Elisseeff and Guyon present a method for visually and quantitatively assessing whether clustered data actually contain structure, and for choosing the number of clusters. The idea is stability: repeatedly perturb the data (by subsampling), cluster each subsample, and measure the distribution of pairwise similarities between the resulting clusterings. High pairwise similarity signals a stable, real clustering pattern; low similarity signals that the imposed cluster count is not supported by the data. The method is algorithm-agnostic and can detect the absence of structure as well as identify an optimal number of clusters, illustrated on artificial and microarray data.
"The method exploits measurements of the stability of clustering solutions obtained by perturbing the data set. Stability is characterized by the distribution of pairwise similarities between clusterings obtained from sub samples of the data. High pairwise similarities indicate a stable clustering pattern."
Clustering's hardest question is not "how do I cluster?" but "is there anything there, and into how many groups?" — and this paper gives one of the most usable answers: if the structure is real, it survives resampling. Framing validity as reproducibility under perturbation is both intuitive and honest, and using the whole distribution of subsample-clustering agreements (rather than a single index) is what makes it diagnostic rather than a point estimate. It is the unsupervised analogue of cross-validation's stability logic and a natural companion to the model-based route to choosing cluster counts on the mixture/latent-class pages. The caveat is well known in hindsight: stability can be misleading when clusters are unequal in size or density, so it is best read as a strong necessary signal rather than a definitive count.