What Welch’s ANOVA does
Welch’s ANOVA compares the means of three or more groups without assuming equal variances. Classic one-way ANOVA pools the variances of all groups. When group SDs differ, especially with unequal group sizes, its p-values can be badly off. Welch’s version weights each group by its precision, n ÷ s², and adjusts the degrees of freedom. Many statisticians now recommend it as the default, just as Welch’s t-test is recommended over Student’s for two groups.
The formulas
For each group i, the weight is wᵢ = nᵢ ÷ sᵢ², and the weighted grand mean is x̄w = Σwᵢx̄ᵢ ÷ Σwᵢ. Then:
- A = Σwᵢ(x̄ᵢ − x̄w)² ÷ (k − 1);
- Λ = Σ(1 − wᵢ ÷ Σw)² ÷ (nᵢ − 1);
- F = A ÷ (1 + 2(k − 2)Λ ÷ (k² − 1));
- degrees of freedom: k − 1 and (k² − 1) ÷ (3Λ).
The p-value comes from Fisher’s F distribution with these, generally non-integer, degrees of freedom.
Example
Three groups of 10, with means 20, 23 and 25 and SDs 4, 5 and 4.5 (the same data as the one-way ANOVA calculator):
- weighted grand mean: 22.42;
- Welch’s F ≈ 3.41, with 2 and about 17.9 degrees of freedom;
- p ≈ 0.056: not significant at α = 0.05.
The classic ANOVA gives F = 3.10 and p ≈ 0.061. With equal group sizes and similar SDs, the two tests agree closely. They diverge when a small group has a large variance.
When the difference matters
- If the smaller groups have the larger SDs, classic ANOVA gives too many false positives.
- If the larger groups have the larger SDs, it becomes too conservative.
- Welch’s ANOVA keeps close to its nominal error rate in both cases.
A ratio of largest to smallest SD above about 2, as shown by the calculator, is a common warning sign.
After a significant result
Follow up with pairwise comparisons that also allow unequal variances, such as the Games–Howell test, or Welch’s t-tests with a Holm correction.
Assumptions
- Groups are independent.
- Data within each group are roughly normal; Welch’s ANOVA is robust to moderate departures, especially with groups of 10 or more.
- Each group needs at least 2 observations; very small groups make the degrees of freedom unstable.
For strongly skewed data or ordinal scores, the Kruskal–Wallis test is an alternative. Note that it also assumes similar distribution shapes.