Joint, Marginal & Conditional Distributions

Every variable so far has lived alone — one die, one coin, one waiting time. Real questions are rarely so tidy. You usually have two things happening at once: a student’s height and their weight, a part’s machine and whether it is defective, an X and a Y. The one little table that holds the chance of every pairing is where we begin. And once you can read that table two ways — by summing it, and by slicing it — the whole topic turns into bookkeeping.

One table, two readings

The joint PMF p(x, y) = P(X = x, Y = y) gives the chance that both happen together. Across the whole table the entries are non-negative and add to one. From it fall the two simpler views.

The marginal is one variable on its own, found by summing out the other — collapse the columns and you have p(x) = Σ_y p(x,y); collapse the rows and you have p(y). The name comes from writing these totals in the margins of the table.

The conditional is Y once X is fixed, found by taking that one row and re-scaling it to total 1: p(y | x) = p(x,y) / p(x). Dividing by the marginal p(x) is exactly the normaliser that turns a single row back into a proper distribution — the same “shrink to the world where the condition holds” move from the conditional lesson. Drag the circles and toggle “Given B” to feel that shrink once more:

Independence — every cell, not just one

X and Y are independent exactly when the joint splits into the product of the two marginals in every cell:

X ⊥ Y   ⇔   p(x, y) = p(x) · p(y)   for ALL (x, y)

This is a strong demand. A single cell where p(x,y) ≠ p(x)·p(y) breaks independence for the whole pair, so a matching cell proves nothing — you check them all, or find one mismatch to rule it out.

Conditional expectation and total expectation

Once you have the conditional p(y|x), its mean is the conditional expectation E[Y | X = x] = Σ_y y · p(y | x). Read as a function of x, the quantity E[Y | X] is itself a random variable, and averaging it over X brings back the plain mean — the law of total expectation:

E[ E[Y | X] ] = E[Y]

It is the “average of the group averages” rule: split the population by X, average Y inside each group, then average those group-means weighted by group size, and you recover E[Y]. A 2025 question handed over a joint setup and asked for E[E[X|Y]], where the entire trick is spotting that it collapses to E[X] with no computation.

Reading a 2×2 table

A joint PMF of (X, Y), each in {0, 1}:

	Y = 0	Y = 1
X = 0	0.10	0.20
X = 1	0.30	0.40

The four cells sum to 1, so it is a valid joint PMF.

Sum out a variable for each marginal — add across the rows for X, down the columns for Y:

p(X=0) = 0.10 + 0.20 = 0.30        p(Y=0) = 0.10 + 0.30 = 0.40
p(X=1) = 0.30 + 0.40 = 0.70        p(Y=1) = 0.20 + 0.40 = 0.60

Now slice. The conditional of Y given X = 1 is that row divided by p(X=1) = 0.7:

p(Y=0 | X=1) = 0.30 / 0.70 = 3/7 ≈ 0.4286
p(Y=1 | X=1) = 0.40 / 0.70 = 4/7 ≈ 0.5714      (the two sum to 1 ✓)

Are X and Y independent? Test the top-left cell: p(X=0, Y=0) = 0.10, but p(X=0)·p(Y=0) = 0.30·0.40 = 0.12. Since 0.10 ≠ 0.12, one mismatch is enough — they are dependent. Finally, with Y a 0/1 variable only its Y=1 term survives, so E[Y | X = 1] = 0·(3/7) + 1·(4/7) = 4/7 ≈ 0.5714 — for a 0/1 variable the conditional expectation is just the conditional chance that Y = 1.

A question to carry forward

We found X and Y here are dependent — but “dependent” is only a yes-or-no. Here is the thread onward: when two variables do move together, can we put a single number on how much, and on whether they rise together or pull in opposite directions?

In one breath

• A joint PMF p(x,y) tabulates P(X=x, Y=y); entries ≥ 0, summing to 1.
• Marginal = sum out the other variable: p(x) = Σ_y p(x,y) (the totals in the margins).
• Conditional = take one slice and re-normalise: p(y|x) = p(x,y)/p(x).
• Independence is strong: p(x,y) = p(x)·p(y) in every cell — one mismatch (0.10 vs 0.12) breaks it.
• Conditional expectation E[Y|X=x] = Σ_y y·p(y|x); the law of total expectation E[E[Y|X]] = E[Y] (average of group averages) collapses a nested expectation with no algebra.

Practice