Skip to content
Omnikalc

Gini impurity and information gain calculator

Enter the positive and negative counts that fall in the left and right branches of a split to get the Gini impurity, the entropy and the gain of the split.

Updated on
Your inputs

Value out of range

Value out of range

Value out of range

Value out of range

Results

Information gain of the split

0.2781 bits

The split makes the branches purer
Gini gain of the split
0.1800
Gini impurity of the parent node
0.5000
Weighted Gini impurity of the children
0.3200
Entropy of the parent node
1.0000 bits
Weighted entropy of the children
0.7219 bits
Positives in the parent node
50.0 %

The Gini impurity and information gain calculator evaluates a binary split in a decision tree: it gives the Gini impurity, the entropy and the gain from the counts of positive and negative examples in each branch.

How the calculation works

  • Enter the positives and negatives in the left branch.
  • Enter the positives and negatives in the right branch.
  • The calculator gives the information gain, the Gini gain, the impurities of the parent and of the children and the entropies.

Example

A split sends 40 positives and 10 negatives to the left and 10 positives and 40 negatives to the right. The parent has 50.0 % positives, a Gini of 0.5000 and an entropy of 1.0000 bit. Each child has 80 % or 20 % positives, so a Gini of 0.32 and an entropy of 0.722 bit. The weighted Gini is 0.3200, a gain of 0.1800, and the information gain is 0.2781 bit.

Why it matters

The calculator helps data science students, analysts and developers who build or explain decision trees and random forests, and who want to check by hand why an algorithm picked a split.

Good to know

The calculation covers two classes and a binary split. With more classes, the Gini is 1 minus the sum of squared proportions, and the entropy is the sum of −p log2 p over the classes. Gain favors splits with many branches, so C4.5 uses a gain ratio, and trees also use a minimum number of samples per leaf. The best split of one step is not necessarily the best tree, since the choice is greedy. The numbers depend on the counts, not only on the proportions: a tiny node may give a large gain by chance, which is why pruning and cross-validation are used. A gain of zero means the split leaves the class mix unchanged in both branches.

The Shannon entropy calculator explains the measure of information, the confusion matrix calculator evaluates classifiers, the sensitivity and specificity calculator covers test performance and the probability calculator handles basic chances.

Frequently asked questions

What are Gini impurity and entropy?

Both measure how mixed the classes are in a node. For two classes with a share p of positives, the Gini impurity is 1 − p² − (1 − p)² and the entropy is −p log2 p − (1 − p) log2 (1 − p). Both are 0 for a pure node, and highest at 50 / 50, where the Gini is 0.5 and the entropy 1 bit.

How is the gain computed?

The gain is the impurity of the parent minus the average impurity of the two children, weighted by their sizes. A decision tree tries many candidate splits and keeps the one with the largest gain, then repeats on each branch.

Gini or entropy: which one to use?

In practice they choose almost the same splits. Gini is slightly cheaper to compute because it needs no logarithm, while information gain comes from information theory and is used in the ID3 and C4.5 algorithms. The default of many libraries is Gini.

Was this calculator helpful?

Thanks for your feedback!

Results are estimates provided for information purposes. They do not replace professional advice.