From the three axioms alone, derive the complement rule, P(empty set) = 0, the rule for an event inside another, the addition rule for overlapping events and the union bound, each with its area picture, then check them on the dice grid with fractions and a simulation.
M06.1.4L1 Core25 minIn review (what this means)
The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.
Derive from the three axioms that the empty event has probability 0, the complement rule, and that an event inside another is no more likely.
Derive the addition rule for two events by cutting their union into three disjoint pieces, and use it on the dice grid.
State and prove the union bound, and use it to cap the chance that at least one of several events happens.
Compute the probability of a union with fractions.Fraction, and check it by simulation with numpy.random.default_rng.
The question
Roll two dice. What is the chance that the sum is 7 or the first die is even? Adding \({\frac{6}{36} + \frac{18}{36}}\) gives the wrong answer. And a model is checked on 20 slices of its test data, each with a 1% chance of a false alarm: how likely is at least one false alarm? Both questions are answered by rules that follow from the three axioms, with no new assumptions.
From the last lesson
1. State the three axioms of probability.
Show answer
\({\Prob(A) \geq 0}\) for every event; \({\Prob(\Omega) = 1}\); and for disjoint events \({A_1, A_2, \dots}\) the probability of the union is the sum of the probabilities.
2. A die is loaded so that a six has probability \({\frac{1}{2}}\) and each other face \({\frac{1}{10}}\). What is P(even)?
Show answer
Add the outcomes: \({0.1 + 0.1 + 0.5 = 0.7}\).
Intuition
Keep the picture of the last lesson: \(\Omega\) is a square of area 1 and a probability is an area. Two events \(A\) and \(B\) that overlap cut their union into three pieces that do not overlap: \(A\) only, both, and \(B\) only.
M06.1.4-F1
The addition rule as areas. The union is three pieces that do not overlap. Adding the areas of \(A\) and \(B\) counts the hatched overlap twice, so take it away once.
The area of \({A \cup B}\) is the sum of its three pieces, 0.42. The areas of \(A\) and \(B\) add to 0.50, which is too much by exactly the overlap, 0.08. So
That is the addition rule. The other rules of this lesson are the same kind of reasoning: cut a region into pieces that do not overlap, and let Axiom 3 add them up.
Build it up
Step 1. Nothing has probability 0. The empty event never occurs, so its area should be 0.
Step 2. Not \(A\) is what is left.\(A\) and its complement \({A^c}\) do not overlap and together fill the square. Their areas must add to 1.
Step 3. Inside means no bigger. If \(A\) lies inside \(B\), then \(B\) is \(A\) plus the extra piece \({B - A}\). That piece cannot have negative area, so \(A\) is no more likely than \(B\).
M06.1.4-F2
Two rules as areas. Left: an event and its complement fill the square. Right: an event inside another is the smaller event plus a leftover piece.
Step 4. Overlaps are counted once. The addition rule of the intuition.
Step 5. A ceiling that needs no overlap. The overlap \({\Prob(A \cap B)}\) is never negative, so dropping it can only make the right side bigger: \({\Prob(A \cup B) \leq \Prob(A) + \Prob(B)}\). This is the union bound. It needs nothing about how \(A\) and \(B\) overlap, which is exactly why it is so useful when nobody knows.
The formal statement
Each rule below is derived from the three axioms and the rules before it. Nothing else is used.
Rule 1. \({\Prob(\emptyset) = 0}\). Take the sequence \({\Omega, \emptyset, \emptyset, \dots}\). No two of its members share an outcome, because \(\emptyset\) has none, so they are disjoint. Their union is \(\Omega\). Axiom 3 and then Axiom 2 give
Subtract 1 from both sides: an endless sum of copies of \({\Prob(\emptyset)}\) equals 0. By Axiom 1, \({\Prob(\emptyset) \geq 0}\). If it were bigger than 0, adding endlessly many copies would grow past any number, not stay at 0. So \({\Prob(\emptyset) = 0}\).
Rule 2. Finite additivity. For disjoint \(A\) and \(B\), apply Axiom 3 to \({A, B, \emptyset, \emptyset,
\dots}\), whose union is \({A \cup B}\). By Rule 1 the empty sets add nothing, so
\[
\begin{aligned}
&\Prob(A \cup B) = \Prob(A) + \Prob(B) \\
&\text{when } A \cap B = \emptyset .
\end{aligned}
\]
The same padding works for any finite list of disjoint events.
Rule 3. The complement rule.\(A\) and \({A^c}\) are disjoint and their union is \(\Omega\). Rule 2 and Axiom 2 give \({\Prob(A) + \Prob(A^c) = \Prob(\Omega) = 1}\), so
\[
\Prob(A^c) = 1 - \Prob(A).
\]
Rule 4. An event inside another. Suppose \({A \subseteq B}\). Then \(B\) is the union of \(A\) and \({B - A}\), which are disjoint. Rule 2 gives \({\Prob(B) = \Prob(A) + \Prob(B - A)}\), and by Axiom 1 the last term is at least 0. So
Since every event lies inside \(\Omega\), this also gives \({\Prob(A) \leq \Prob(\Omega) = 1}\): every probability lies between 0 and 1. That was a consequence, not an axiom.
Rule 5. The addition rule. For any events \(A\) and \(B\), cut the union into three disjoint pieces:
\[
\begin{aligned}
A \cup B ={}& (A \cap B^c) \cup (A \cap B) \\
&\cup (A^c \cap B).
\end{aligned}
\]
An outcome in the union is in \(A\) only, in both, or in \(B\) only, and in exactly one of the three. Now \(A\) itself is the first two pieces, and \(B\) is the last two, so by Rule 2
Add the two lines. The right side holds all three pieces once, plus \({\Prob(A \cap B)}\) a second time, and the three pieces add up to \({\Prob(A \cup B)}\):
The union bound. The three areas add to 0.46, but the union covers only 0.39: the overlaps were added twice. The sum is always at least the union.
Go deeper: the union bound for n events, by induction
For one event there is nothing to prove. Suppose the bound holds for some \({n \geq 1}\), and add an event \({A_{n+1}}\). Write \({U = A_1 \cup \cdots \cup A_n}\). The two-event bound, then the bound for \(n\) events, give
So it holds for \({n + 1}\) events, and by induction (M01.6.4) for every \(n\).
Where the union bound is used. Suppose 20 checks are run on a model, and each one, on its own, raises a false alarm with probability 0.01. Nobody knows how the checks overlap. The union bound still says that the chance of at least one false alarm is at most \({20 \times 0.01 = 0.2}\). Turned around, to keep that chance at most 0.05, run each check at \({0.05 / 20 = 0.0025}\): the Bonferroni correction for testing many hypotheses at once. Learning theory leans on the same step: if each of \(k\) candidate models could mislead you with probability at most \(\delta\), then the chance that any one of them does is at most \({k\delta}\).
Worked example
Two fair dice. Find P(the sum is 7 or the first die is even).
Name the events.\(A\) = “the sum is 7” and \(B\) = “the first die is even”. Each of the 36 outcomes has probability \({\frac{1}{36}}\), so the probability of an event is its number of cells over 36.
Count each piece.\(A\) has 6 cells, the diagonal from \({(1, 6)}\) to \({(6, 1)}\). \(B\) has 3 even columns of 6 cells, 18 in all. The overlap \({A \cap B}\) holds the cells on the diagonal whose first die is even: \({(2, 5)}\), \({(4, 3)}\) and \({(6, 1)}\), three cells.
M06.1.4-F4
The worked example on the grid. The hatched cells belong to both events, so counting \(A\) and then \(B\) meets them twice.
Check with the complement. The union misses the cells that are in neither: an odd first die (1, 3 or 5) and a sum other than 7. Each odd column has 6 cells, one of which has sum 7, so 5 per column and 15 in all. Then \({\Prob(A \cup B) = 1 - \frac{15}{36} = \frac{21}{36}}\), the same answer.
Code
Exact first, with fractions, on the 36 outcomes.
from fractions import Fraction as Ffrom itertools import productomega =list(product(range(1, 7), repeat=2))def P(event):"""Each of the 36 outcomes has probability 1/36: add them up."""return F(len(event), len(omega))A = {w for w in omega if w[0] + w[1] ==7}B = {w for w in omega if w[0] %2==0}print("P(A) =", P(A), " P(B) =", P(B), " P(A and B) =", P(A & B))print("P(A or B) by counting:", P(A | B), " by the rule:", P(A) + P(B) - P(A & B))print("by the complement:", 1- P(set(omega) - (A | B)))
P(A) = 1/6 P(B) = 1/2 P(A and B) = 1/12
P(A or B) by counting: 7/12 by the rule: 7/12
by the complement: 7/12
Now a simulation with NumPy’s random generator. rng.integers(1, 7, ...) draws whole numbers from 1 up to 6, because the upper end is left out. The share of rolls in \({A \cup B}\) should be close to \({\frac{7}{12} \approx 0.583}\).
import numpy as nprng = np.random.default_rng(12)rolls = rng.integers(1, 7, size=(100_000, 2))first, second = rolls[:, 0], rolls[:, 1]in_A = first + second ==7in_B = first %2==0print("share in A or B:", (in_A | in_B).mean().round(4), " exact:", round(7/12, 4))print("counts:", in_A.sum(), "+", in_B.sum(), "-", (in_A & in_B).sum(), "=", (in_A | in_B).sum())
share in A or B: 0.5828 exact: 0.5833
counts: 16743 + 49916 - 8374 = 58285
The second line holds exactly, not just roughly: the addition rule is true for counts too, because it is really a fact about counting overlaps. Last, both rules on every pair of events of three fair coin flips, \({256 \times 256}\) pairs, with each of the 8 outcomes given \({\frac{1}{8}}\).
from itertools import combinationsflips = ["".join(p) for p in product("HT", repeat=3)]events = [set(c) for k inrange(9) for c in combinations(flips, k)]Pf =lambda e: F(len(e), 8)worst =0for E in events:for G in events:assert Pf(E | G) == Pf(E) + Pf(G) - Pf(E & G) # the addition ruleassert Pf(E | G) <= Pf(E) + Pf(G) # the union bound worst =max(worst, Pf(E) + Pf(G) - Pf(E | G))print(len(events) **2, "pairs checked; the bound is off by at most", worst)
65536 pairs checked; the bound is off by at most 1
The bound can be off by a whole 1: for \({E = G = \Omega}\) it says \({\Prob(\Omega) \leq 2}\). That is the price of knowing nothing about the overlap.
Explore
M06.1-X1
Event explorer
Choose events \(A\) and \(B\) from the presets, or click cells to make your own, then light up their union, intersection or a complement. The read-out counts the cells and checks the addition rule. Try “at least one six” against “doubles”: they share one cell.
Pitfalls
M06.1.4-F5
Wrong beside right. “A six on the first die” and “a six on the second” share the roll \({(6, 6)}\), so adding their probabilities counts it twice.
“The chance of at least one six in two rolls is \({\frac{1}{6} + \frac{1}{6} = \frac{1}{3}}\).”
The two events overlap in \({(6, 6)}\). The addition rule gives \({\frac{6}{36} + \frac{6}{36} - \frac{1}{36} = \frac{11}{36}}\). The complement rule agrees: “no six at all” holds \({5 \times 5 = 25}\) of the 36 rolls, and \({1 - \frac{25}{36} = \frac{11}{36}}\). Push the mistake further, to six rolls, and it claims a six is certain: \({6 \times \frac{1}{6} = 1}\).
“The union bound gives the probability of the union.”
It gives a ceiling, and a ceiling can be far too high. For six rolls of a die, “a six on roll \(k\)” has probability \({\frac{1}{6}}\) for each \(k\), so the bound says P(at least one six) is at most 1, which says nothing. Counting, the subject of module M06.2, gives the exact answer: of the \({6^6}\) equally likely sequences of six rolls, \({5^6}\) have no six, so the chance is \({1 - (\frac{5}{6})^6}\), about 0.665. The bound is close only when the events hardly overlap.
“P(A) + P(not A) is 1 only for nice events.”
It holds for every event, because \(A\) and \({A^c}\) are always disjoint and always fill \(\Omega\). That is what makes “at least one” problems easy: work out “none”, then subtract from 1.
Check yourself
Recall
1. Derive the complement rule from the axioms in one line.
Show answer
\(A\) and \({A^c}\) are disjoint with union \(\Omega\), so \({\Prob(A) + \Prob(A^c) = \Prob(\Omega) = 1}\).
2. State the addition rule for two events, and say why the last term is there.
Show answer
\({\Prob(A \cup B)} = {\Prob(A)} + {\Prob(B)} - {\Prob(A \cap B)}\). Adding \({\Prob(A)}\) and \({\Prob(B)}\) counts the overlap twice, so it is taken away once.
3. Why is \({\Prob(\emptyset) = 0}\)?
Show answer
\(\Omega\) is the union of the disjoint events \({\Omega, \emptyset, \emptyset, \dots}\), so Axiom 3 gives \({1 = 1 + \Prob(\emptyset) + \Prob(\emptyset) + \cdots}\). That endless sum of equal terms that are not negative can be 0 only if each term is 0.
4. State the union bound. When is it an equality?
Show answer
\({\Prob(A \cup B) \leq \Prob(A) + \Prob(B)}\), and for \(n\) events the probability of the union is at most the sum. For two events it is an equality exactly when \({\Prob(A \cap B) = 0}\), for example when they are disjoint.
5. On two dice, what is P(the sum is 7 or the first die is even)?
1.★\({\Prob(A) = 0.5}\), \({\Prob(B) = 0.4}\) and \({\Prob(A \cap B) = 0.15}\). Find \({\Prob(A \cup B)}\) and the probability that neither happens.
Show solution
\({\Prob(A \cup B)} = 0.5 + 0.4 - 0.15 = 0.75\). “Neither” is \({(A \cup B)^c}\) by De Morgan’s first law, so its probability is \({1 - 0.75 = 0.25}\).
2.★★ Can two events with \({\Prob(A) = 0.7}\) and \({\Prob(B) = 0.6}\) be disjoint? What is the smallest their overlap can be?
Show solution
No. If they were disjoint, Rule 2 would give \({\Prob(A \cup B) = 1.3}\), but every probability is at most 1. Rearranging the addition rule,
because \({\Prob(A \cup B) \leq 1}\). The overlap has probability at least 0.3, and 0.3 can happen: on the unit square, let \(A\) be the strip from 0 to 0.7 across and \(B\) the strip from 0.4 to 1. They overlap in the strip from 0.4 to 0.7, of area 0.3, and together they cover the whole square.
3.★★★ Derive the addition rule for three events,
\[
\begin{aligned}
&\Prob(A \cup B \cup C) \\
&\quad = \Prob(A) + \Prob(B) + \Prob(C) \\
&\qquad - \Prob(A \cap B) - \Prob(A \cap C) \\
&\qquad - \Prob(B \cap C) \\
&\qquad + \Prob(A \cap B \cap C),
\end{aligned}
\]
and use it on two dice with \(A\) = “sum is 7”, \(B\) = “first die even” and \(C\) = “a double”.
Show solution
Treat \({A \cup B}\) as one event and use the two-event rule:
\[
\begin{aligned}
&\Prob(A \cup B \cup C) \\
&\quad = \Prob(A \cup B) + \Prob(C) \\
&\qquad - \Prob\big((A \cup B) \cap C\big).
\end{aligned}
\]
An outcome is in \({(A \cup B) \cap C}\) exactly when it is in \(C\) and in \(A\) or \(B\), that is, in \({A \cap C}\) or in \({B \cap C}\). So \({(A \cup B) \cap C} = {(A \cap C)} \cup {(B \cap C)}\), and the two-event rule once more, with \({(A \cap C) \cap (B \cap C) = A \cap B \cap C}\), gives
\[
\begin{aligned}
&\Prob\big((A \cup B) \cap C\big) \\
&\quad = \Prob(A \cap C) + \Prob(B \cap C) \\
&\qquad - \Prob(A \cap B \cap C).
\end{aligned}
\]
Put this and \({\Prob(A \cup B)} = {\Prob(A)} + {\Prob(B)} - {\Prob(A \cap B)}\) into the first line to get the formula.
On the dice: \({|A| = 6}\), \({|B| = 18}\), \({|C| = 6}\); \({|A \cap B| = 3}\); \({A \cap C = \emptyset}\) (a double has an even sum); \({B \cap C = \{(2, 2), (4, 4), (6, 6)\}}\), 3 cells; and \({A \cap B \cap C = \emptyset}\). So
Sources. Each was opened and checked against this lesson on 14 November 2026.
Hossein Pishro-Nik, Introduction to Probability, Statistics, and Random Processes, Section 1.3.3 Finding Probabilities (open access): the complement rule, \({\Prob(\emptyset) = 0}\), \({\Prob(A - B)} = {\Prob(A)} - {\Prob(A \cap B)}\), inclusion and exclusion for two events, and \({\Prob(A) \leq \Prob(B)}\) when \(A\) lies in \(B\). It gets \({\Prob(\emptyset) = 0}\) from the complement rule; this lesson proves it first, straight from Axiom 3, so that the rule for two disjoint events rests on nothing unproved. Its Section 6.2.1, The Union Bound and Extension, states the union bound for \(n\) events and builds it up by induction.
Charles M. Grinstead and J. Laurie Snell, Introduction to Probability, second edition (free PDF). Section 1.2: Theorem 1.1 (an event inside another, the complement) and Theorem 1.4, \({\Prob(A \cup B)} = {\Prob(A)} + {\Prob(B)} - {\Prob(A \cap B)}\), proved by counting how often each outcome is added.
Jeremy Orloff and Jonathan Bloom, Reading 2: Probability, terminology and examples, MIT OpenCourseWare 18.05, Spring 2022. Section 3: the complement rule, the rule for disjoint events and inclusion and exclusion, each drawn as a Venn diagram, with the overlap “counted twice”.
NumPy, numpy.random.Generator.integers: the upper end high is left out unless endpoint=True, so integers(1, 7) draws 1 to 6.
