What is ANOVA, and why use it instead of multiple t-tests?
ANOVA (Analysis of Variance) tests whether the means of three or more groups are simultaneously equal by partitioning total variance into between-group and within-group components. Running multiple pairwise t-tests instead inflates the family-wise Type I error rate, which ANOVA avoids by using a single omnibus F-test.
How to think about it
When comparing more than two groups, ANOVA is the standard first step. It answers “are any group means different?” — not which ones — keeping Type I error controlled at alpha.
The core idea: variance decomposition
Total variance in the data breaks into two components:
- Between-group variance (MSB): How much the group means vary around the grand mean. High MSB suggests real differences.
- Within-group variance (MSW): How much observations vary within each group. This is pure noise from individual variation.
The F-statistic is simply their ratio:
F = MSB / MSW = (SS_between / df_between) / (SS_within / df_within)
Under H0 (all group means equal), F follows an F-distribution with (k-1, N-k) degrees of freedom, where k is the number of groups and N is the total sample size. A large F means the between-group signal is large relative to within-group noise.
Why not k*(k-1)/2 pairwise t-tests?
With k = 5 groups, you would run 10 t-tests. At alpha = 0.05 each, the probability of at least one false positive across all 10 tests is 1 - (0.95)^10 ≈ 0.40. ANOVA keeps the single omnibus Type I error at alpha regardless of k.
Assumptions
- Independence of observations.
- Normality within each group (robust to violations for large n by CLT).
- Homogeneity of variances across groups (Levene’s test checks this; Welch’s ANOVA relaxes it).
Post-hoc testing
A significant F-test only tells you some means differ. To identify which pairs, run post-hoc tests with multiple-comparison correction: Tukey’s HSD, Bonferroni, or Scheffe, depending on whether comparisons are pre-planned or exploratory.
Two-way ANOVA
Extends one-way ANOVA to two categorical factors and their interaction. Tests three hypotheses simultaneously: main effect of factor A, main effect of B, and A*B interaction.