Bayes' theorem in plain English

The last lesson left a sharp edge: we flipped a conditional — turned “given the segment, how likely is a click?” into “given a click, how likely the segment?” — and warned that doing it carelessly springs the base-rate trap. Here is the formula that flips it correctly, every time. Its one-sentence story is far simpler than its reputation: you believe something, new information arrives, and you update what you believe. That is the whole of Bayes’ theorem — the math is merely how to do that update precisely, base rate and all.

The formula, said plainly

In words:

• P(hypothesis) — your prior (what you believed before seeing the evidence).
• P(evidence | hypothesis) — the likelihood (how probable the evidence is if the hypothesis is true).
• P(evidence) — the marginal (how probable the evidence is across all hypotheses; just a normalizing constant).
• P(hypothesis | evidence) — the posterior (your updated belief after seeing the evidence).

Why does the formula flip the conditional from P(evidence | hypothesis) to P(hypothesis | evidence)? Because what you can measure from data is how often each hypothesis would produce this kind of evidence; Bayes rescales those measurements by the prior and turns them into the thing you actually want — a probability over hypotheses. The denominator just ensures everything still sums to 1.

See the update happen

Before any numbers, build the intuition by hand. Below is a coin of unknown bias θ. Your belief about θ starts as a prior (the faint dashed curve), and every flip is evidence that reshapes it into a posterior (the bold curve). Watch how heads pull the belief right, tails pull it left, and more data makes the curve narrow and confident.

Notice two things. The posterior always sits between your prior and the observed head-rate — it’s a compromise. And the more you flip, the less your starting prior matters: set “No idea” and “Believe fair,” flip fifty times each, and both arrive at nearly the same answer. That convergence is why Bayesians and frequentists usually agree once the data is plentiful.

Worked example: the spam filter

You’re building an email classifier. From your training data:

• 30% of emails are spam: P(spam) = 0.3
• The word “free” appears in 60% of spam emails: P("free" | spam) = 0.6
• The word “free” appears in 5% of legitimate emails: P("free" | not spam) = 0.05

A new email arrives containing the word “free.” What’s the probability it’s spam?

# Bayes for the spam filter, written line by line.

# Priors (what we knew before seeing the email)
p_spam     = 0.30
p_not_spam = 0.70

# Likelihoods -- how 'free' behaves in each class
p_free_given_spam     = 0.60
p_free_given_not_spam = 0.05

# Marginal -- total probability of seeing 'free' across all emails
# = P(free | spam)*P(spam) + P(free | not spam)*P(not spam)
p_free = p_free_given_spam * p_spam + p_free_given_not_spam * p_not_spam

# Bayes' theorem
p_spam_given_free = (p_free_given_spam * p_spam) / p_free

print(f"Prior belief it's spam:       {p_spam:.2%}")
print(f"After seeing 'free':          {p_spam_given_free:.2%}")
print(f"That's a {p_spam_given_free/p_spam:.2f}x update.")

Prior belief it's spam:       30.00%
After seeing 'free':          83.72%
That's a 2.79x update.

Before seeing “free,” you’d have called it spam 30% of the time. After seeing “free,” you’d call it spam ~84% of the time. The word “free” is strong evidence — it’s much more common in spam than in normal mail — and Bayes turns that evidence into an updated probability.

Crucially: a 60% likelihood does not mean “60% chance it’s spam.” Likelihood and posterior are different things. The prior matters too.

What happens when the prior changes

Same email, same word, but suppose only 5% of your mail is spam (lucky you).

def bayes_spam(p_spam, p_free_given_spam=0.6, p_free_given_not_spam=0.05):
    p_not_spam = 1 - p_spam
    p_free = p_free_given_spam * p_spam + p_free_given_not_spam * p_not_spam
    return (p_free_given_spam * p_spam) / p_free

for prior in [0.05, 0.30, 0.50, 0.80]:
    posterior = bayes_spam(prior)
    print(f"P(spam) prior = {prior:.0%}   →   P(spam | 'free') = {posterior:.2%}")

P(spam) prior = 5%   →   P(spam | 'free') = 38.71%
P(spam) prior = 30%   →   P(spam | 'free') = 83.72%
P(spam) prior = 50%   →   P(spam | 'free') = 92.31%
P(spam) prior = 80%   →   P(spam | 'free') = 97.96%

Same evidence, very different conclusions. With a 5% prior, even after seeing “free” you’re only 39% sure it’s spam. With an 80% prior, you’re 98% sure. The same evidence updates different starting beliefs differently. That’s the heart of Bayesian thinking.

This is also why base rates matter so much in medicine: a 99%-accurate test for a rare disease still produces mostly false positives, because the prior P(disease) is so low that even a small P(positive | healthy) dominates.

Updating with more evidence

If you see two words — say “free” and “winner” — and you assume they’re conditionally independent given the class, the math just multiplies the likelihoods. That assumption is exactly what makes the algorithm “naive Bayes”:

# Naive Bayes for two words
p_spam = 0.30

# Likelihoods (probabilities of each word given the class)
p_free_spam,   p_free_ham   = 0.60, 0.05
p_winner_spam, p_winner_ham = 0.40, 0.01

# Combined likelihood under each class (naive: multiply word probs)
lik_spam = p_free_spam * p_winner_spam * p_spam
lik_ham  = p_free_ham  * p_winner_ham  * (1 - p_spam)

# Normalize to a posterior
posterior_spam = lik_spam / (lik_spam + lik_ham)
print(f"After 'free' AND 'winner':   P(spam | ev) = {posterior_spam:.4%}")

After 'free' AND 'winner':   P(spam | ev) = 99.5162%

Two suggestive words together drive the posterior to 99.5% — far past either word alone — because the naive step multiplies their likelihoods. “Naive” because the independence assumption is rarely true — words co-occur. But it’s surprisingly hard to beat for text classification, which is why naive Bayes was the workhorse spam filter for a couple of decades.

Where else Bayes shows up

• A/B testing (Bayesian flavor) — instead of a p-value, you maintain a posterior over each variant’s conversion rate and ask “what’s the probability B is better than A?”
• Recommendation and search — many ranking signals are P(click | features), Bayes-updated as new clicks arrive.
• Reinforcement learning — Bayesian RL agents maintain a posterior over which actions are best and use it to balance exploration vs exploitation.
• LLMs — autoregressive next-token prediction is literally P(next_token | previous_tokens). Sampling from that posterior is what generation is.

In one breath

Bayes’ theorem flips a conditional the right way: P(hyp|ev) = P(ev|hyp)·P(hyp) / P(ev) — the posterior is the likelihood times the prior, over the marginal (a normaliser). It is a recipe for updating belief: start with a prior, multiply in how well each hypothesis explains the evidence, renormalise. The prior is load-bearing — the same word “free” makes spam 39% likely under a 5% prior but 98% under an 80% one, and a 99%-accurate test for a 1-in-1000 disease still leaves a positive only ~9% likely, because the base rate dominates. Stack independent evidence by multiplying likelihoods (the “naive” in naive Bayes), and the very same update sits under spam filters, Bayesian A/B tests, RL exploration, and an LLM’s P(next token | previous).

Practice

A question to carry forward

Every Bayesian update here ran on a handful of single numbers — P(spam) = 0.3, a likelihood of 0.6. But look again at the coin demo: your belief about the bias θ was never a number, it was a whole curve — a prior shaped like one bell, reshaped by data into a narrower posterior. Even the spam filter’s “60% of spam contains free” is a summary of a whole distribution of words. Probabilities, it turns out, usually arrive as shapes.

So here is the thread onward: what are the handful of named distributions — the bell, the count, the waiting-time, the single coin-flip — that priors, likelihoods, and raw data actually take in practice? Which real situation does each one model, what is the difference between a discrete PMF and a continuous PDF (and how can a density be greater than 1?), and how do you sample any of them in one line of NumPy so you can simulate the systems you cannot solve by hand?