The BLEU score calculator gives the BLEU score, the brevity penalty, the geometric mean of the precisions, the length ratio, the score without the penalty and the points lost to it, from the four n-gram precisions and the lengths.
How the calculation works
- Enter the modified precision for 1-grams, 2-grams, 3-grams and 4-grams, as percentages.
- Enter the length of the candidate and of the reference translation, in words.
- Read the BLEU score, from 0 to 100.
Example
With precisions of 75 %, 55 %, 40 % and 30 % percent, the geometric mean is 47.17 %. A candidate of 100 words words against a reference of 105 words has a brevity penalty of 0.9512, so BLEU is 44.87, instead of 47.17, which loses 2.30 points.
Why it matters
The calculator helps students and engineers in natural language processing who want to understand how the score is built, to check a result from an evaluation tool or to see what a higher precision or a longer output would do to it.
Good to know
The precisions must come from a counting tool run on the test set, with the same tokenization as the reference, and the lengths are summed over the whole corpus for a corpus-level score. A zero precision at any order makes the plain BLEU zero, and the calculator floors it at a small value to avoid this. BLEU rewards exact word overlap and does not recognize synonyms or reordering, so a good translation can score low. It is best used to compare versions of the same system on the same data, and not to rank different language pairs. Report the tokenization and the number of references with the score, since both change it by several points.
Related tools
The confusion matrix calculator evaluates classification, the sensitivity and specificity calculator measures a test, the geometric mean calculator averages values multiplicatively and the percentage calculator computes shares.