The simple ("naive") Bayesian classifier assumes attributes are independent given the class, yet performs surprisingly well even when that assumption is clearly false. Domingos and Pazzani explain why: although the classifier's probability estimates are only optimal (under quadratic loss) when independence holds, the classifier itself can be optimal under zero-one loss (misclassification rate) even when independence is violated by a wide margin. They show the region of quadratic-loss optimality is an infinitesimal fraction of the region of zero-one optimality, so naive Bayes has a far wider range of applicability than previously believed.
"The classifier itself can be optimal under zero-one loss (misclassification rate) even when [the independence] assumption is violated by a wide margin. The region of quadratic-loss optimality of the Bayesian classifier is in fact a second-order infinitesimal fraction of the region of zero-one optimality."
The paper that explained a long-standing embarrassment: naive Bayes "shouldn't" work — its independence assumption is almost always false — yet it is a perennially strong baseline. Domingos and Pazzani's resolution is elegant and general: classification is an argmax, and an argmax is robust to a great deal of error in the underlying probabilities, so a model can be a poor density estimator and an excellent classifier at the same time. That decoupling of calibration from decision is a genuinely useful lesson well beyond naive Bayes — it is why crude models often classify well and why proper-scoring-rule accuracy (Brier) and misclassification rate can disagree. On the wiki it rounds out the classifier cluster beside SVMs and trees, and its argument is a clean companion to the bias–variance intuition behind why simple models generalize.