The Gini impurity and information gain calculator evaluates a binary split in a decision tree: it gives the Gini impurity, the entropy and the gain from the counts of positive and negative examples in each branch.
How the calculation works
- Enter the positives and negatives in the left branch.
- Enter the positives and negatives in the right branch.
- The calculator gives the information gain, the Gini gain, the impurities of the parent and of the children and the entropies.
Example
A split sends 40 positives and 10 negatives to the left and 10 positives and 40 negatives to the right. The parent has 50.0 % positives, a Gini of 0.5000 and an entropy of 1.0000 bit. Each child has 80 % or 20 % positives, so a Gini of 0.32 and an entropy of 0.722 bit. The weighted Gini is 0.3200, a gain of 0.1800, and the information gain is 0.2781 bit.
Why it matters
The calculator helps data science students, analysts and developers who build or explain decision trees and random forests, and who want to check by hand why an algorithm picked a split.
Good to know
The calculation covers two classes and a binary split. With more classes, the Gini is 1 minus the sum of squared proportions, and the entropy is the sum of −p log2 p over the classes. Gain favors splits with many branches, so C4.5 uses a gain ratio, and trees also use a minimum number of samples per leaf. The best split of one step is not necessarily the best tree, since the choice is greedy. The numbers depend on the counts, not only on the proportions: a tiny node may give a large gain by chance, which is why pruning and cross-validation are used. A gain of zero means the split leaves the class mix unchanged in both branches.
Related tools
The Shannon entropy calculator explains the measure of information, the confusion matrix calculator evaluates classifiers, the sensitivity and specificity calculator covers test performance and the probability calculator handles basic chances.