Skip to content
Omnikalc

BLEU score calculator

Enter the modified n-gram precisions from 1 to 4 and the lengths of the candidate and reference to get the BLEU score and the brevity penalty.

Updated on
Your inputs
%

Value out of range

%

Value out of range

%

Value out of range

%

Value out of range

words

Value out of range

words

Value out of range

Results

BLEU score (0 to 100)

44.87

Brevity penalty
0.9512
Geometric mean of the precisions
47.17 %
Length ratio (candidate / reference)
0.952
BLEU if the length matched the reference
47.17
Points lost to the brevity penalty
2.30

The BLEU score calculator gives the BLEU score, the brevity penalty, the geometric mean of the precisions, the length ratio, the score without the penalty and the points lost to it, from the four n-gram precisions and the lengths.

How the calculation works

  • Enter the modified precision for 1-grams, 2-grams, 3-grams and 4-grams, as percentages.
  • Enter the length of the candidate and of the reference translation, in words.
  • Read the BLEU score, from 0 to 100.

Example

With precisions of 75 %, 55 %, 40 % and 30 % percent, the geometric mean is 47.17 %. A candidate of 100 words words against a reference of 105 words has a brevity penalty of 0.9512, so BLEU is 44.87, instead of 47.17, which loses 2.30 points.

Why it matters

The calculator helps students and engineers in natural language processing who want to understand how the score is built, to check a result from an evaluation tool or to see what a higher precision or a longer output would do to it.

Good to know

The precisions must come from a counting tool run on the test set, with the same tokenization as the reference, and the lengths are summed over the whole corpus for a corpus-level score. A zero precision at any order makes the plain BLEU zero, and the calculator floors it at a small value to avoid this. BLEU rewards exact word overlap and does not recognize synonyms or reordering, so a good translation can score low. It is best used to compare versions of the same system on the same data, and not to rank different language pairs. Report the tokenization and the number of references with the score, since both change it by several points.

The confusion matrix calculator evaluates classification, the sensitivity and specificity calculator measures a test, the geometric mean calculator averages values multiplicatively and the percentage calculator computes shares.

Frequently asked questions

What is the BLEU score?

BLEU, for bilingual evaluation understudy, is a metric for the quality of machine translation. It compares a candidate translation with one or more human references by the overlap of n-grams, sequences of one to four words, and gives a score between 0 and 1, often written between 0 and 100.

How is it calculated?

The score is the brevity penalty times the geometric mean of the modified n-gram precisions, usually for n from 1 to 4 with equal weights: BLEU = BP × exp(¼ Σ ln p_n). The penalty is 1 when the candidate is longer than the reference, and exp(1 − r/c) when it is shorter, which stops the system from gaining by giving very short outputs.

What is a modified precision?

It is the share of the n-grams of the candidate that appear in the reference, where each n-gram is counted at most as many times as it appears in the reference. This stops a candidate that repeats one correct word from getting a perfect score.

How should the score be read?

The score depends on the language pair, the number of references and the tokenization, so values from different test sets cannot be compared. It correlates with human judgment on a corpus of sentences, but it is a poor guide for one sentence, and it does not measure meaning or fluency. Modern evaluations add neural metrics and human review.

Was this calculator helpful?

Thanks for your feedback!

Results are estimates provided for information purposes. They do not replace professional advice.