What is the difference between parametric and non-parametric tests, and when should you prefer one over the other?
Parametric tests assume the data follow a specific distribution (usually normal) and make inferences about distributional parameters like the mean. Non-parametric tests make no such distributional assumption and typically operate on ranks. Non-parametric tests are more robust but less powerful when parametric assumptions genuinely hold.
How to think about it
The parametric vs non-parametric choice is not about sample size alone. It is about whether the distributional assumptions are defensible given the data-generating process.
Parametric tests
Assume the population follows a specific distribution (most commonly normal) with unknown parameters. Statistical inference focuses on those parameters (mean, variance).
Common examples: t-test, z-test, ANOVA, Pearson correlation, linear regression.
Strengths: More powerful when assumptions hold; directly interpretable in terms of means; naturally handles multiple covariates (regression frameworks).
Key assumptions: Independent observations; specific distributional form; often equal variance across groups.
Non-parametric tests
Make minimal distributional assumptions. Most rank the data and test hypotheses about the median or the overall distribution.
Common examples and their parametric equivalents:
| Non-parametric | Parametric equivalent |
|---|---|
| Mann-Whitney U (Wilcoxon rank-sum) | Two-sample t-test |
| Wilcoxon signed-rank | Paired t-test |
| Kruskal-Wallis | One-way ANOVA |
| Spearman correlation | Pearson correlation |
Strengths: Robust to outliers; valid for ordinal data; no normality assumption needed; appropriate when n is very small.
Trade-off: Lower statistical power than the parametric equivalent when the normality assumption does hold. Also harder to extend to complex multi-variable settings.
Decision guide
- Data are continuous and approximately normal (or
n > 30and no severe skew): use parametric. - Data are ordinal, heavily skewed, or have severe outliers that cannot be removed: use non-parametric.
- Small n (under 15) and normality cannot be verified: use non-parametric.
- Comparing two independent groups with extreme outliers in production metrics (e.g., revenue per user): Mann-Whitney U is often preferred over t-test.