Joint, Marginal & Conditional Distributions

Every variable so far has lived alone:

• one die
• one coin
• one waiting time

Real questions are rarely so tidy. You usually have two things happening at once, such as:

• a student’s height and their weight
• a part’s machine and whether it is defective
• an X and a Y

The one little table that holds the chance of every pairing is where we begin. Once you can read that table two ways — by summing it, and by slicing it — the whole topic turns into bookkeeping.

One table, two readings

The joint PMF — probability mass function, the rule that assigns a chance to each possible value — is written p(x, y) = P(X = x, Y = y). It gives the chance that both happen together.

Across the whole table the entries are non-negative and add to one. From it fall the two simpler views.

The marginal is one variable on its own, found by summing out the other. Collapse the columns and you have p(x) = Σ_y p(x,y); collapse the rows and you have p(y).

The name comes from writing these totals in the margins of the table.

The conditional is Y once X is fixed. Find it by taking that one row and re-scaling it to total 1: p(y | x) = p(x,y) / p(x).

Dividing by the marginal p(x) is exactly the normaliser that turns a single row back into a proper distribution. It is the same “shrink to the world where the condition holds” move from the conditional lesson. Drag the circles and toggle “Given B” to feel that shrink once more:

Independence — every cell, not just one

X and Y are independent exactly when the joint splits into the product of the two marginals in every cell:

X ⊥ Y   ⇔   p(x, y) = p(x) · p(y)   for ALL (x, y)

This is a strong demand. A single cell where p(x,y) ≠ p(x)·p(y) breaks independence for the whole pair.

A matching cell proves nothing. You check them all, or find one mismatch to rule it out.

Conditional expectation and total expectation

Once you have the conditional p(y|x), its mean is the conditional expectation E[Y | X = x] = Σ_y y · p(y | x).

Read as a function of x, the quantity E[Y | X] is itself a random variable. Averaging it over X brings back the plain mean — the law of total expectation:

E[ E[Y | X] ] = E[Y]

It is the “average of the group averages” rule:

1. Split the population by X.
2. Average Y inside each group.
3. Average those group-means weighted by group size.
4. Recover E[Y].

A 2025 question handed over a joint setup and asked for E[E[X|Y]]. The entire trick is spotting that it collapses to E[X] with no computation.

An expectation of an expectation reads like a typo the first time, so unpick it slowly: the two Es are averaging over different things. The inner one is not mysterious — fix X = x.

Then E[Y | X = x] is a plain number, the average of Y inside that one group. Now let x vary. Those numbers, one per group, form a new random variable called E[Y | X].

It takes the X = 0 group’s average whenever X = 0, and the X = 1 group’s average whenever X = 1. It is random for the single reason that X is.

The outer E averages that variable, weighting each group’s average by how likely that group is. This is why the law is not surprising at all once stated in words: carve a population into groups, average inside each group, then average the group averages by group size.

You are back to the overall average you started from.

Reading a 2×2 table

A joint PMF of (X, Y), each in {0, 1}:

	Y = 0	Y = 1
X = 0	0.10	0.20
X = 1	0.30	0.40

The four cells sum to 1, so it is a valid joint PMF.

Marginals

Sum out a variable for each marginal — add across the rows for X, down the columns for Y:

p(X=0) = 0.10 + 0.20 = 0.30        p(Y=0) = 0.10 + 0.30 = 0.40
p(X=1) = 0.30 + 0.40 = 0.70        p(Y=1) = 0.20 + 0.40 = 0.60

Conditional slice

Now slice. The conditional of Y given X = 1 is that row divided by p(X=1) = 0.7:

p(Y=0 | X=1) = 0.30 / 0.70 = 3/7 ≈ 0.4286
p(Y=1 | X=1) = 0.40 / 0.70 = 4/7 ≈ 0.5714      (the two sum to 1 ✓)

Independence and conditional expectation

Are X and Y independent? Test the top-left cell: p(X=0, Y=0) = 0.10, but p(X=0)·p(Y=0) = 0.30·0.40 = 0.12.

Since 0.10 ≠ 0.12, one mismatch is enough — they are dependent. Finally, with Y a 0/1 variable only its Y=1 term survives.

So E[Y | X = 1] = 0·(3/7) + 1·(4/7) = 4/7 ≈ 0.5714. For a 0/1 variable the conditional expectation is just the conditional chance that Y = 1.

Since this table is on the page, use it to watch the law of total expectation actually close. The other group gives E[Y | X = 0] = 0.20 / 0.30 = 2/3.

Now average the two group means, weighted by how big each group is:

E[ E[Y|X] ] = p(X=0)·(2/3) + p(X=1)·(4/7)
            = 0.30·(2/3)   + 0.70·(4/7)
            = 0.20         + 0.40          =  0.60

And E[Y] read straight off the marginals is 0·0.40 + 1·0.60 = 0.60. The two agree exactly, as the law promises.

That is why a question asking for E[E[Y|X]] is asking for E[Y] and nothing more.

A question to carry forward

We found X and Y here are dependent — but “dependent” is only a yes-or-no.

Here is the thread onward: when two variables do move together, can we put a single number on how much, and on whether they rise together or pull in opposite directions?

In one breath

• A joint PMF p(x,y) tabulates P(X=x, Y=y); entries ≥ 0, summing to 1.
• Marginal = sum out the other variable: p(x) = Σ_y p(x,y) (the totals in the margins).
• Conditional = take one slice and re-normalise: p(y|x) = p(x,y)/p(x).
• Independence is strong: p(x,y) = p(x)·p(y) in every cell — one mismatch (0.10 vs 0.12) breaks it.
• Conditional expectation E[Y|X=x] = Σ_y y·p(y|x); the law of total expectation E[E[Y|X]] = E[Y] (average of group averages) collapses a nested expectation with no algebra.

Practice