What is the Law of Large Numbers and how does it differ from the Central Limit Theorem?
The Law of Large Numbers (LLN) says the sample mean converges to the true mean as sample size grows — it is a statement about where the mean lands. The Central Limit Theorem says the sampling distribution of the mean is approximately normal — it is a statement about the shape of that distribution. LLN guarantees convergence; CLT characterises the rate and shape of that convergence.
How to think about it
Distinguish the two versions of LLN, then contrast it cleanly with the CLT — interviewers use this question to check whether candidates confuse “the mean gets close” with “the distribution becomes normal.”
Two versions of the LLN
Weak Law (WLLN): For any ε > 0,
P( |X_bar_n - mu| > epsilon ) → 0 as n → ∞
The probability that the sample mean deviates from μ by more than any fixed ε goes to zero. This is convergence in probability.
Strong Law (SLLN):
P( lim_{n→∞} X_bar_n = mu ) = 1
The sample mean converges to μ with probability 1 — almost sure convergence. The strong law is harder to prove but makes a stronger statement: not just that deviation becomes unlikely, but that the sequence of means eventually stays close to μ forever.
Both require finite mean (E[|X|] < ∞). The SLLN additionally requires finite variance in its most common proof, though it actually holds under the weaker assumption of finite mean alone.
LLN vs CLT: what each says
| Question | Law of Large Numbers | Central Limit Theorem |
|---|---|---|
| What does it say? | X̄ₙ → μ | X̄ₙ ~ N(μ, σ²/n) for large n |
| What it tells you | Where the estimate lands | Shape and rate of sampling error |
| Requires | Finite mean | Finite variance |
| Output | Point convergence | Distribution approximation |
Practical consequences
- Simulation: The LLN justifies Monte Carlo estimation — simulate enough trials and the sample average converges to the true expectation.
- Insurance: An insurer covering many independent, identical risks can predict average payouts accurately (LLN), even though any individual claim is uncertain.
- Gambler’s fallacy: The LLN does not say past outcomes “balance out.” After 100 coin flips landing heads, the next flip is still 50/50. The LLN says the proportion converges, not that future outcomes compensate for past ones.