What statistical power is
The power of a test is the probability that it detects an effect that really exists. A study with 50% power misses a true effect half of the time, which wastes resources and fills the literature with inconclusive results. The usual standard is 80% power, sometimes 90% for confirmatory trials. The complement, β = 1 − power, is the risk of a type II error (a false negative).
Power depends on four quantities: the effect size, the sample size, the significance level α and whether the test is one- or two-tailed. Fix three, and the fourth follows.
The formulas
For a two-sample t-test with n subjects per group and Cohen’s effect size d:
- noncentrality: δ = d × √(n ÷ 2);
- power ≈ P(T > t_crit − δ) with 2n − 2 degrees of freedom, plus the tiny opposite tail in a two-tailed test.
Sample size per group for a target power:
n ≈ 2 × ((z₁₋α/₂ + z_power) ÷ d)² + z₁₋α/₂² ÷ 4
The last term is a small-sample correction that makes the result match exact software such as G*Power to within one subject.
Example
Expected effect d = 0.5 (medium), 64 subjects per group, α = 0.05 two-tailed:
- δ = 0.5 × √32 ≈ 2.83;
- power ≈ 80%, so β ≈ 20%;
- 64 per group (128 in total) is exactly what is needed for 80% power;
- with 64 per group, the smallest effect detectable with 80% power is d ≈ 0.50.
Choosing the effect size
The effect size is the hardest input. Options:
- the smallest effect that would matter in practice, the best choice;
- an effect from previous studies, remembering that published effects tend to be inflated;
- Cohen’s conventions, as a last resort: d = 0.2 small, 0.5 medium, 0.8 large.
Halving the effect size roughly quadruples the sample size: d = 0.25 needs about 253 per group for 80% power.
Good practice
- Compute power before collecting data. “Post hoc” power computed from the observed effect adds nothing to the p-value.
- Plan for dropouts: if you expect 15% attrition, divide the required n by 0.85.
- Unequal groups, paired designs and other tests need different formulas; for paired data, the effect size is the mean difference divided by the SD of the differences.