Expectation, Variance & SD

The last lesson left us with a whole table of chances and a wish: could just two numbers stand in for it — one for where the variable sits, one for how much it swings? They can. Picture a simple game: roll a fair die and win that many rupees. Roll a six, win six; roll a one, win one. Before you play, what would you expect to win on a typical roll?

The first number: what you expect on average

You cannot win 3.5 rupees on any single roll, yet that is the honest long-run average — add the six equally likely payouts and share them out, (1+2+3+4+5+6)/6 = 3.5. That probability-weighted average is the expectation, written E[X]. When the chances are not equal, you weight each value by its own mass instead of sharing equally:

E[X] = Σ x · p(x)

— run through the values, multiply each by its chance, and add. That is the first of our two summary numbers.

Expectation has one habit worth trusting completely: it is linear. Scaling and shifting the variable scales and shifts its average, exactly — E[aX + b] = aE[X] + b — and this holds always, even when variables depend on each other. Double every payout and add two, and the average doubles and adds two.

The second number: how much it swings

The average alone hides a lot. A game paying 3 or 4 rupees and a game paying 0 or 7 can share the very same average, yet feel completely different to play. The variance captures that swing: the average squared distance from the mean. The form you actually compute from is the mean of the squares minus the square of the mean:

Var(X) = E[X²] − (E[X])²

Two formulas just went past for the same thing, and being handed both without a word is where variance starts to feel slippery. They are not rivals. The definition is Var(X) = E[(X − μ)²] — literally average the squared distances from the mean. Expand that square and the second form falls out:

E[(X − μ)²] = E[X² − 2μX + μ²] = E[X²] − 2μ·E[X] + μ² = E[X²] − 2μ² + μ² = E[X²] − μ²

Same quantity, algebra already done. The reason everyone computes with the second version is practical: it needs one sweep through the table, whereas the definition makes you find μ first and then sweep again to measure distances from it. A free consequence falls out too — since a variance can never be negative, E[X²] is always at least (E[X])², and the gap between them is the variance.

The two pieces are genuinely different computations — E[X²] weights each x² by its chance, while (E[X])² squares the single number E[X]. Take the square root and you return to the original units: the standard deviation, SD = √Var(X). Spread reacts to scaling and shifting differently from the mean — Var(aX + b) = a²·Var(X) — because a shift b slides every value together and changes nothing about the swing, while a scale a stretches every distance and so comes out squared. If Var(X) = 2, then Var(5X − 1) = 5²·2 = 50: the −1 contributes nothing.

Watching it on the die

A fair die shows 1 to 6, each with chance 1/6. Find E[X], E[X²], Var(X), and SD.

Average the faces, then average the squared faces, then subtract the square of the mean:

E[X]   = (1+2+3+4+5+6) / 6        = 21/6  = 3.5
E[X²]  = (1+4+9+16+25+36) / 6     = 91/6  ≈ 15.1667

Var(X) = E[X²] − (E[X])²
       = 15.1667 − 3.5²
       = 15.1667 − 12.25
       = 2.9167                   (exactly 35/12)

SD     = √2.9167                  ≈ 1.708

Mind the order: square each face and average for E[X²] = 15.1667, then subtract the square of the mean, 3.5² = 12.25. Squaring at the wrong moment is the usual slip.

A question to carry forward

We now have a way to squeeze any table into two numbers, but we still computed them by grinding through the whole list. Here is the thread onward: a few special shapes of randomness — a single yes/no trial, a count of successes — come up again and again, and for those the mean and variance have ready-made formulas you will not need to re-derive. What are those repeat-offender distributions, and what are their two numbers?

In one breath

• Expectation E[X] = Σ x·p(x) — the probability-weighted average (the long-run mean); fair die = 3.5.
• Linearity E[aX+b] = aE[X]+b holds always; but E[g(X)] ≠ g(E[X]) for nonlinear g — E[X²] ≠ (E[X])², E[1/X] ≠ 1/E[X].
• Variance Var(X) = E[X²] − (E[X])² (mean of squares − square of mean); SD = √Var, back in X’s units.
• Affine variance Var(aX+b) = a²·Var(X) — a shift b leaves the swing untouched; a scale a comes out squared.
• Fair die: E[X]=3.5, Var = 35/12 ≈ 2.917, SD ≈ 1.708.

Practice