Skip to content
Omnikalc

Brier score calculator

Enter, for three forecast probability levels, the number of forecasts and how often the event happened to get the Brier score, its decomposition into reliability, resolution and uncertainty, and the skill score against always forecasting the base rate.

Updated on
Your inputs
%

Value out of range

Value out of range

%

Value out of range

%

Value out of range

Value out of range

%

Value out of range

%

Value out of range

Value out of range

%

Value out of range

Results

Brier score

0.1700

Moderate accuracy
Reliability (calibration error, lower is better)
0.0044
Resolution (ability to separate cases, higher is better)
0.0717
Uncertainty (base rate × (1 − base rate))
0.2373
Brier skill score against the base rate
0.2837
Real skill beyond the base rate
Base rate of the event
38.8 %
Total number of forecasts
200

The Brier score calculator measures the accuracy of probability forecasts from three forecast levels, with the decomposition into reliability, resolution and uncertainty and the skill score.

How the calculation works

  • Enter, for each of the three levels, the forecast probability, the number of forecasts and the observed frequency of the event.
  • The calculator weights the levels by their number.
  • It gives the Brier score, reliability, resolution, uncertainty, the skill score and the base rate.

Example

A forecaster issued 100 forecasts of 10 % (event seen 15 %), 50 of 50 % (45 %) and 50 of 90 % (80 %). The base rate is 38.8 %, so the uncertainty is 0.2373. Reliability is 0.0044 and resolution 0.0717, so the Brier score is 0.0044 − 0.0717 + 0.2373 = 0.1700. The skill score is 0.2837: the forecasts add real information over the base rate, with room to improve calibration.

Why it matters

The calculator helps students of forecasting and machine learning, meteorologists, analysts who build probability models for risk or sports, and anyone who keeps score of predictions.

Good to know

Grouping forecasts into a few levels gives a decomposition that is exact only when all forecasts in a group share the same probability; with real data, use the ungrouped Brier score and bins that are wide enough to hold many cases. A good Brier score depends on the base rate: forecasting rare events is easy to score well on by always forecasting a low number, so look at the skill score. The Brier score is a proper score, meaning that honest probabilities give the best expected value. It is not suited to extremely rare events, where the log score may be better. Report the number of forecasts, since small samples are noisy.

The confusion matrix calculator evaluates yes-or-no predictions, the logistic regression calculator produces probabilities, the Matthews correlation calculator summarizes binary classification and the KL divergence calculator compares distributions.

Frequently asked questions

What is the Brier score?

It is the mean squared difference between the forecast probability and the outcome, coded 1 if the event happened and 0 if not. It ranges from 0 (perfect) to 1 (always confidently wrong). Forecasting 50 % every time scores 0.25.

What do reliability and resolution mean?

The decomposition is BS = reliability − resolution + uncertainty. Reliability measures calibration: do events happen as often as forecast? Resolution measures whether the forecasts separate days or cases with different frequencies. Uncertainty depends only on the base rate.

What is the skill score?

It compares the Brier score with that of a reference that always forecasts the base rate: skill = 1 − BS / uncertainty. Positive values mean the forecaster adds information, and negative values mean worse than the climate average.

Was this calculator helpful?

Thanks for your feedback!

Results are estimates provided for information purposes. They do not replace professional advice.