Skip to content
Omnikalc

Mutual information calculator: how many bits does X tell you about Y?

Enter the counts of a 2 × 2 table to get the mutual information in bits, the entropies of both variables and the normalised mutual information.

Updated on
Your inputs

Value out of range

Value out of range

Value out of range

Value out of range

Results

Mutual information

0.1912 bits

Entropy of X
1.0000 bits
Entropy of Y
0.9928 bits
Normalised mutual information
0.1919
Share of the uncertainty about Y removed by knowing X
19.3 %

Two variables can be linked in ways that a correlation coefficient misses, and information theory has a measure for any kind of link. The mutual information calculator takes the four counts of a 2 × 2 table and gives the mutual information in bits, the entropy of X, the entropy of Y, the normalised mutual information and the share of the uncertainty about Y removed by knowing X.

How the calculation works

  • Fill in the 2 × 2 table of counts for two yes-or-no variables X and Y: both present, only X, only Y and neither.
  • The calculator turns the counts into joint and marginal probabilities.
  • The mutual information is I(X;Y) = Σ p(x,y) × log₂( p(x,y) / (p(x) × p(y)) ), measured in bits. It is 0 when X and Y are independent.
  • Read the entropies of X and Y, the normalised mutual information, which runs from 0 to 1, and the share of the uncertainty about Y that disappears once you know X.

Example

Out of 100 cases, X and Y are both present in 40, only X in 10, only Y in 15 and neither in 35. X is present half of the time, so its entropy is 1.000 bit, and Y is present 55 % of the time, so its entropy is 0.993 bit. The mutual information is 0.1912 bit. The normalised value is 0.192, and knowing X removes 19.3 % of the uncertainty about Y.

Why it matters

Mutual information is used to pick the most informative features in machine learning, to measure how much a symptom tells about a disease, how much a word predicts a topic, or how much a signal carries through a noisy channel. It gives a unit, the bit, that is easy to compare across problems.

Good to know

The calculator handles two binary variables from a table of counts, with the plug-in estimate, which is biased upward for small samples. It does not test for significance, and it treats the counts as exact. For continuous variables or more categories, you must bin the data first, and the result depends on the binning. Use it as a descriptive measure.

The KL divergence calculator measures the gap between two distributions, the chi-square test calculator tests independence in a table, the correlation significance calculator tests a linear link, and the confidence interval calculator measures uncertainty.

Frequently asked questions

What is mutual information?

It measures how much knowing one variable reduces the uncertainty about another. It is zero when the two variables are independent, and it grows as they become more dependent. Unlike correlation, it captures any kind of relationship, not only a straight line, and it is measured in bits when the logarithm has base 2.

How does it relate to entropy?

The mutual information is the entropy of Y minus the entropy of Y once X is known, and it is symmetric: I(X;Y) = H(X) + H(Y) − H(X,Y). It can never exceed the smaller of the two entropies. When it equals H(Y), knowing X tells you Y completely.

What is normalised mutual information?

It rescales the mutual information to a value between 0 and 1 by dividing by the geometric mean of the two entropies, which makes it easier to compare across variables with different entropies. It is common in clustering evaluation and in feature selection. A value near 0 means independence, and a value near 1 means that one variable determines the other.

What are its limits with small samples?

Mutual information estimated from counts is biased upward in small samples, because random chance always creates some apparent dependence. With few observations, a positive value can come from noise alone. For tests of independence, compare the result with a chi-square test or with permutations, and prefer large samples.

Was this calculator helpful?

Thanks for your feedback!

Results are estimates provided for information purposes. They do not replace professional advice.