What is Simpson's paradox? Walk through a concrete example.
Simpson's paradox occurs when a trend that appears in several subgroups disappears or reverses when those subgroups are combined. It arises because a lurking variable (the group size itself, correlated with both treatment and outcome) distorts the aggregate.
How to think about it
Simpson’s paradox is the phenomenon where an association present in every subgroup reverses in the combined data. It is not a mathematical contradiction — it is a signal that group membership is a confounder that must be controlled.
Worked numeric example — hospital survival rates
Two hospitals treat the same two conditions: mild cases and severe cases.
Hospital A
| Condition | Survived | Total | Rate |
|---|---|---|---|
| Mild | 800 | 1 000 | 80% |
| Severe | 200 | 1 000 | 20% |
| Combined | 1 000 | 2 000 | 50% |
Hospital B
| Condition | Survived | Total | Rate |
|---|---|---|---|
| Mild | 90 | 100 | 90% |
| Severe | 400 | 900 | 44% |
| Combined | 490 | 1 000 | 49% |
Within each condition, Hospital B has a higher survival rate (90% vs 80% for mild; 44% vs 20% for severe). Yet the combined rate favours Hospital A (50% vs 49%).
The reason: Hospital A handles proportionally more mild cases (1 000 out of 2 000 = 50%), which inflate its aggregate. Hospital B is disproportionately sent severe cases (900 out of 1 000 = 90%), dragging its aggregate down. Condition severity is the lurking confounder.
Inline diagram — within-group vs aggregate trend
Why it happens mathematically
The aggregate rate is a weighted average of subgroup rates where the weights (group sizes) differ between A and B. When groups with systematically different base rates also differ in size across treatments, the weights distort the aggregate.