Brier (1950) Verification of Forecasts Expressed in Terms of Probability

Summary

Writing at the US Weather Bureau, Brier tackles a long-controversial problem: how to verify forecasts that are stated as probabilities. He proposes a score — now the Brier score — that measures the mean squared difference between forecast probabilities and the realized outcomes (1 if the event occurred, 0 otherwise), summed over the mutually exclusive forecast categories. The score rewards forecasters for issuing calibrated, honest probabilities and became the founding proper scoring rule for probabilistic forecast verification.

Key Claims

• Score probability forecasts by squared error. For $N$ occasions and $r$ mutually exclusive categories, $P=\frac1N\sum_{i=1}^N\sum_{j=1}^r (f_{ij}-E_{ij})^2$, where $f_{ij}$ is the forecast probability of category $j$ on occasion $i$ and $E_{ij}=1$ if category $j$ occurred (else 0). For a single binary event this is $\frac1N\sum_i (f_i-o_i)^2$.
• Bounded and oriented. The score ranges from 0 (perfect) to 2 in the $r$-category form (0 to 1 in the binary MSE form); smaller is better.
• Encourages honesty. Because squared error is minimized in expectation by reporting one's true probability, the rule discourages hedging — it is (in later terminology) a strictly proper scoring rule.
• Resolves the "goodness scale" problem. It provides an objective, non-arbitrary scale for comparing probabilistic forecasts, sidestepping the disputes over categorical verification scores.

Concepts Introduced or Extended

• Proper Scoring Rule

Entities Mentioned

• Glenn W. Brier

Quotes

"One of the greatest arguments raised against forecast verification is that forecasts which may be the 'best' according to the accepted system of arbitrary scores may not be the most useful forecasts."

My Take

A three-page weather-bureau note that quietly founded the field of probabilistic forecast evaluation. Its genius is simplicity: score a probability by its squared distance from the 0/1 outcome, and the arithmetic does the rest — the expected score is minimized by telling the truth, so the metric and the incentive align. That single idea underlies modern proper scoring rules, the reliability–resolution–uncertainty decomposition (Murphy), and the whole apparatus of forecast calibration used far beyond meteorology (elections, medicine, machine-learning classifiers). It is the natural companion to the calibration and interval-forecast material on the wiki, and a reminder that the most durable statistical tools are often the plainest.