Conditional & Total Probability

Step outside one morning not knowing last night’s weather, and the chance the ground is wet might be one in three. Now someone tells you it rained. In that instant the chance leaps to almost certain. Notice that nothing about the ground itself changed in that moment — only what you know changed. Re-judging a chance in the light of something you have just learned is the whole of this lesson.

Putting the re-judging on paper

We do this re-judging in our heads all day. To do it carefully we need a name and a rule. The chance of A once we know B has happened is written P(A | B), read “the probability of A given B”. And the rule is this: knowing B shrinks the world down to just the cases where B is true, and inside that smaller world we ask what fraction also have A.

P(A | B) = P(A ∩ B) / P(B),     for P(B) > 0

The denominator P(B) is the shrunk-down world; the numerator P(A ∩ B) is the slice of it that also has A. Multiply the rule across and it rearranges into the multiplication rule, often the handier form:

P(A ∩ B) = P(A | B) · P(B)

— the chance both happen equals the chance B happens, times the chance of A once B has. It chains to more events the same way, one condition at a time: P(A ∩ B ∩ C) = P(A)·P(B | A)·P(C | A ∩ B), which is exactly what you do when you walk down a probability tree, multiplying along the path.

Drag the two circles below and turn on “Given B”: everything outside B fades, and you are left re-reading the chance of A inside the shrunken B-world. That fading is what “conditioning” is.

Building a chance out of cases

Sometimes you cannot read P(A) off directly, but you can split the world into a few exhaustive, non-overlapping cases and find A’s chance inside each. Then you recombine, weighting each case by how likely it was to begin with.

Picture a factory whose parts come from two machines. Machine 1 makes 60% of the parts and 2% of its parts are defective; Machine 2 makes the other 40% and 5% of its parts are defective. What is the chance a part picked at random is defective? Walk the tree: down each branch to the machine, then on to “defective”, multiplying as you go.

That recombining is the law of total probability: if cases B₁, B₂, … are exhaustive and non-overlapping, P(A) = Σ P(A | Bᵢ) · P(Bᵢ). Here it gives 0.6·0.02 + 0.4·0.05 = 0.012 + 0.020 = 0.032 — a 3.2% defect rate, built from two cases.

The two conditionals are not the same

Now a warning that the whole next lesson rests on. We were given P(defective | Machine 2) = 0.05. But flip the question — given that a part is defective, what is the chance it came from Machine 2? That is P(Machine 2 | defective), a different number entirely. Divide that machine’s path by the total: 0.020 / 0.032 ≈ 0.625. So P(defective | M2) = 0.05 while P(M2 | defective) = 0.625 — the two conditionals point in opposite directions and must never be swapped.

A question to carry forward

We just turned P(defective | machine) into P(machine | defective) by dividing one path by the total. Here is the thread for the next lesson: that move — flipping a conditional around using the total-probability denominator — has a name and a single clean formula. What is the rule that takes any P(B | A) and hands you back P(A | B)?

In one breath

• Conditioning re-judges a chance once you know B: shrink to the B-world, then P(A|B) = P(A∩B)/P(B).
• Multiplication rule: P(A∩B) = P(A|B)·P(B); it chains — P(A∩B∩C) = P(A)·P(B|A)·P(C|A∩B) (walk the tree).
• Law of total probability: split into exhaustive disjoint cases, recombine — P(A) = Σ P(A|Bᵢ)·P(Bᵢ) (factory defect = 0.6·0.02 + 0.4·0.05 = 0.032).
• The trap: P(A|B) ≠ P(B|A) — P(defective|M2)=0.05 but P(M2|defective)=0.625. Flipping that conditional is Bayes, next.
• “Without replacement” draws are dependent, so the second chance is a conditional, not the first.

Practice