\( \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\mat}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M06.3 · Conditional probability and independence

A free, illustrated module of 4 lessons and one interactive in M06 Probability I: Conditional probability: shrinking the sample space; The multiplication rule and probability trees; The law of total probability; Independence.

M06.3 L1 · Core 4 lessons · about 100 min Live · in review

Part of M06 Probability I in the Mathematics track.

New information changes probabilities. These four lessons show what learning that an event happened does to every other probability: it throws away the outcomes that are ruled out and rescales the rest. From that one definition come the multiplication rule and the probability tree, the law of total probability, which builds a probability from its pieces, and independence, the case in which learning tells you nothing. Two dice, an inbox with spam and a medical test carry the numbers throughout.

What you need first. Sample spaces, events and the axioms of probability (M06.1), and counting equally likely outcomes, including draws without replacement (M06.2). The code uses plain Python with fractions and itertools, a little NumPy for simulation and one call to pandas.crosstab.

Lessons

1. Conditional probability: shrinking the sample space M06.3.1 25 min

   Learning that an event B happened throws away every outcome outside B and shares the whole probability among the outcomes that are left. Define P(A | B), see why the order around the bar matters, and compute it on two dice and an inbox.

2. The multiplication rule and probability trees M06.3.2 25 min

   Turn the definition of conditional probability round and you get the multiplication rule: the chance of A and B is the chance of B times the chance of A given B. Chain it over several stages, draw it as a probability tree, and redo a mini-batch draw one example at a time.

3. The law of total probability M06.3.3 25 min

   Split the sample space into pieces, find the chance of an event inside each piece, weight each by the size of its piece and add. Derive the law of total probability, see it as adding strips of area or leaves of a tree, and use it on an inbox and a medical test.

4. Independence M06.3.4 25 min

   Two events are independent when learning one leaves the chance of the other where it was. Define independence by a product, prove it matches the conditional picture, tell pairwise from mutual independence, and meet conditional independence, the assumption behind every likelihood that is a product.

The interactive

M06.3-X1 · Zoom-in square. A unit square cut into a strip for B and a strip for not B, each cut by the share of A in it. Set P(B), P(A | B) and P(A | not B) with sliders or by dragging the cuts, and watch the four areas, the matching probability tree and P(A) change together. The read-out says when the two shares are level, which is exactly when A and B are independent. It lives in The law of total probability.