What the sign test does
The sign test is the simplest test for paired data or for a median. It ignores the size of the differences and only keeps their direction: plus when the second measurement is higher (or the value is above the hypothesized median), minus when it is lower. Ties are dropped. If there is no real effect, plus and minus are equally likely, so the number of plus signs follows a binomial distribution with p = ½.
Because it relies on so little, it works with almost any data: ordinal ratings, preferences between two products, or measurements with extreme outliers.
The exact p-value
With n non-tied pairs and k plus signs:
- two-tailed p = 2 × P(X ≤ min(k, n − k)), capped at 1;
- right-tailed p = P(X ≥ k);
- left-tailed p = P(X ≤ k);
where X follows Binomial(n, ½). The calculator computes these sums exactly. It also shows the normal approximation z = (|k − n/2| − 0.5) ÷ (√n ÷ 2) for comparison, which is accurate once n exceeds about 25.
Example
Twelve tasters compare two recipes: 10 prefer the new one and 2 the old one.
- P(X ≤ 2) = (1 + 12 + 66) ÷ 4,096 ≈ 0.0193;
- two-tailed p ≈ 0.039: significant at α = 0.05;
- the normal approximation gives p ≈ 0.043, close to the exact value.
When to use it
- Paired comparisons where only the direction is reliable, such as “better or worse”.
- Testing a median: count the values above and below the hypothesized median.
- Preference tests between two options.
Limits
- It throws away information about the size of the differences, so it has less power than the Wilcoxon signed-rank test or the paired t-test when those apply. On normal data, the sign test needs roughly 50% more pairs to reach the same power as the t-test.
- Many ties reduce the effective sample size, since ties are removed.
- The pairs must be independent: several measurements from the same person do not count as separate pairs.
The sign test is also the basis of the exact McNemar test, which applies the same binomial logic to the discordant pairs of a 2 × 2 table.