Skip to main content

Turning a conditional around

Knowing how often spam contains ‘free’ is not the same as knowing how often a ‘free’ email is spam. Derive Bayes’ rule in two lines, name its four parts, and turn one conditional probability into the other on the inbox.
M06.4.1L1 Core25 min
In review

The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

Before this lesson: M06.3 Conditional probability and independence

What you will be able to do
  • Derive Bayes’ rule in two lines from the definition of conditional probability.
  • Name the prior, the likelihood, the evidence and the posterior in a problem, and find the evidence with the law of total probability.
  • Compute a posterior by hand, on a probability tree read backwards, and with a short Python function.
  • Explain why P(A | B) and P(B | A) differ, and say when they are equal.

The question

A spam filter learns from emails that people have already labelled. Counting is easy in that direction: 60% of the spam contains the word ‘free’. But when a new email arrives with ‘free’ in it, the filter needs the opposite number: how likely is this email to be spam? The two numbers are not the same. How do you get from one to the other?

Intuition

The course’s inbox has 100 emails: 40 are spam and 60 are not. The word ‘free’ appears in 24 of the spam emails and in 6 of the others. Lay the 100 emails out as a grid, with spam on the left, and fill in every email that contains ‘free’.

Two copies of a 10 by 10 grid of 100 small squares, one per email. The four left columns, labelled spam, are blue; the other six columns, labelled not spam, are sky blue. Filled squares contain the word free: the top six rows of the four spam columns (24 squares) and the top six squares of the fifth column (6 squares). In the left panel, titled Look inside the spam, a thick frame surrounds the four spam columns and the rest is faded; underneath, P of free given spam equals 24 over 40 equals 0.6. In the right panel, titled Look inside the free emails, a thick frame surrounds the 30 filled squares and the rest is faded; underneath, P of spam given free equals 24 over 30 equals 0.8.
M06.4.1-F1

Two questions about the same grid. On the left we look inside the spam and ask how much of it says ‘free’. On the right we look inside the ‘free’ emails and ask how much of them is spam. The 24 emails that are both are the same in each picture; only the total they are divided by changes.

A conditional probability (M06.3.1) keeps only the outcomes in the condition and asks what share of them has the property. So the two questions use the same overlap and different totals:

  • \({\Prob(\text{free} \mid \text{spam}) = 24/40 = 0.6}\): the overlap divided by all the spam.
  • \({\Prob(\text{spam} \mid \text{free}) = 24/30 = 0.8}\): the overlap divided by all the ‘free’ emails.

Turning a conditional around means keeping the overlap and swapping the total. Bayes’ rule is that sentence written with probabilities instead of counts, so that it works when you do not have a grid to count, only the rates.

Build it up

Step 1. What we know runs forwards. A filter is trained on labelled email, so it knows how each kind of email behaves. In probabilities: \({\Prob(\text{spam}) = 0.4}\), \({\Prob(\text{free} \mid \text{spam}) = 0.6}\) and \({\Prob(\text{free} \mid \text{not spam}) = 0.1}\). Each of these starts from the kind of email and asks about the word.

Step 2. What we want runs backwards. The question starts from the word and asks about the kind: \({\Prob(\text{spam} \mid \text{free})}\).

Step 3. The overlap can be built from either end. Picture the inbox as a unit square whose area is probability. Cut it first into a spam strip of width 0.4 and a not-spam strip of width 0.6, then cut each strip by its share of ‘free’. Or cut it first by ‘free’ and then by spam. Both routes end at the same piece of area.

Two unit squares. In the left one, titled Cut by spam first, a vertical line splits the square into a spam strip of width 0.4 and a not-spam strip of width 0.6. Inside the spam strip the top 0.6 of its height is a yellow block labelled spam and free, 0.24; below it is spam, no free. The not-spam strip has a thin band of free at the top, labelled 0.06. Under it: P of spam times P of free given spam equals 0.4 times 0.6 equals 0.24. In the right one, titled Cut by free first, a horizontal line splits off a top band of height 0.3 for free; 0.8 of its width is the same yellow block, spam and free, 0.24, and the rest is 0.06. Under it: P of free times P of spam given free equals 0.3 times 0.8 equals 0.24.
M06.4.1-F2

One piece of area, measured two ways. On the left it is a width times a height, \({0.4 \times 0.6}\); on the right it is \({0.3 \times 0.8}\). Both give 0.24.

On the left, the multiplication rule of M06.3.2 gives the area as a product, and on the right the same rule with the roles swapped gives the same area as another product:

\[ \begin{aligned} &\Prob(\text{spam and free}) \\ &\quad= \Prob(\text{spam})\,\Prob(\text{free} \mid \text{spam}) \\ &\quad= \Prob(\text{free})\,\Prob(\text{spam} \mid \text{free}). \end{aligned} \]

Step 4. Set the two equal and divide. The two products are equal, so dividing by \({\Prob(\text{free})}\) leaves the number we want. The only piece we were not given is \({\Prob(\text{free})}\), and the law of total probability (M06.3.3) supplies it: add the ‘free’ area in the spam strip to the ‘free’ area in the other strip, \({0.24 + 0.06 = 0.30}\). Then \({\Prob(\text{spam} \mid \text{free})} = {0.24 / 0.30} = {0.8}\), the same answer the grid gave.

The formal statement

NoteBayes’ rule

For events \(H\) and \(E\) with \({\Prob(H) > 0}\) and \({\Prob(E) > 0}\),

\[ \cmodel{\Prob(H \mid E)} = \frac{\cdata{\Prob(E \mid H)}\;\cparam{\Prob(H)}}{\Prob(E)} . \]

Derivation. The definition of conditional probability, read with \(E\) as the condition, says

\[ \Prob(H \mid E) = \frac{\Prob(H \cap E)}{\Prob(E)} . \]

The same definition with \(H\) as the condition says \({\Prob(E \mid H) = \Prob(E \cap H) / \Prob(H)}\). Multiply both sides by \(\Prob(H)\), and use that \({E \cap H}\) and \({H \cap E}\) are the same event:

\[ \Prob(H \cap E) = \Prob(E \mid H)\,\Prob(H) . \]

Put this into the numerator of the first line, and Bayes’ rule is proved. That is all: two uses of one definition.

The letters stand for hypothesis and evidence, and each part has a name.

  • The prior \(\cparam{\Prob(H)}\) is how probable the hypothesis is before the evidence.
  • The likelihood \(\cdata{\Prob(E \mid H)}\) is how probable the evidence is if the hypothesis is true.
  • The evidence \(\Prob(E)\) is how probable the evidence is overall.
  • The posterior \(\cmodel{\Prob(H \mid E)}\) is how probable the hypothesis is after the evidence.
Bayes' rule drawn as boxes. On the left an orange-red box titled posterior holds P of spam given free and the value 0.8. An equals sign leads to a fraction. Its numerator is a blue box titled likelihood, P of free given spam, 0.6, times an orange box titled prior, P of spam, 0.4. Below a thick fraction bar is a grey box titled evidence, P of free, 0.30. Above the boxes: 0.6 times 0.4 over 0.30 equals 0.24 over 0.30 equals 0.8. Below: P of free equals 0.6 times 0.4 plus 0.1 times 0.6 equals 0.30, the evidence comes from the law of total probability.
M06.4.1-F3

The four parts of Bayes’ rule, filled in with the inbox. The colours match the equation above.

The evidence by total probability. The hypothesis \(H\) and its complement \(H^c\) split \(\Omega\) into two pieces, so the law of total probability gives \(\Prob(E) = {\Prob(E \mid H)\,\Prob(H)} + {\Prob(E \mid H^c)\,\Prob(H^c)}\). Put that into the denominator and every term is something we usually know:

\[ \begin{aligned} \Prob(H \mid E) &= \frac{a}{a + b}, \quad\text{where} \\ a &= \Prob(E \mid H)\,\Prob(H), \\ b &= \Prob(E \mid H^c)\,\Prob(H^c). \end{aligned} \]

The numerator \(a\) is one of the two terms of the denominator. So the posterior is a share: the part of the evidence that comes through \(H\), out of all the evidence.

When are the two conditionals equal? Write the multiplication rule both ways, \({\Prob(H \mid E)\,\Prob(E)} = {\Prob(E \mid H)\,\Prob(H)}\), and divide (when \({\Prob(H \cap E) > 0}\)):

\[ \frac{\Prob(H \mid E)}{\Prob(E \mid H)} = \frac{\Prob(H)}{\Prob(E)} . \]

The two directions differ by the ratio of the two totals. They are equal exactly when \({\Prob(H) = \Prob(E)}\). In the inbox, \({0.8 / 0.6 = 0.4 / 0.3 = 4/3}\).

Go deeper: what a zero in Bayes' rule means

If \({\Prob(E) = 0}\) the rule says nothing, because \({\Prob(H \mid E)}\) is not defined: we cannot condition on something that cannot happen. If \({\Prob(E \mid H) = 0}\) and \({\Prob(E) > 0}\), the numerator is 0 and so is the posterior: evidence that is impossible under a hypothesis rules it out completely. Lesson M06.4.4 uses this when a die shows a 5 and the four-sided die drops out. The other zero is a warning. If \({\Prob(H) = 0}\), the posterior is 0 whatever the evidence, so a prior of exactly 0 can never be revised. That is why people who build models avoid priors of exactly 0 or 1.

Worked example

The inbox, by Bayes’ rule. We are given the prior \({\Prob(\text{spam}) = 0.4}\) and the two likelihoods \({\Prob(\text{free} \mid \text{spam}) = 0.6}\) and \({\Prob(\text{free} \mid \text{not spam}) = 0.1}\). Draw the tree forwards, multiply along each path, then read it backwards.

A probability tree drawn left to right. From a box labelled inbox, a branch marked 0.4 leads to a blue box spam and a branch marked 0.6 to a sky-blue box not spam. From spam, branches 0.6 and 0.4 lead to the leaves free: 0.4 times 0.6 equals 0.24 and no free: 0.4 times 0.4 equals 0.16. From not spam, branches 0.1 and 0.9 lead to free: 0.6 times 0.1 equals 0.06 and no free: 0.6 times 0.9 equals 0.54. The two free leaves are yellow. On the right, under the heading Backwards, arrows from the two yellow leaves meet at a box P of free equals 0.24 plus 0.06 equals 0.30, and an arrow leads down to an orange-red box P of spam given free equals 0.24 over 0.30 equals 0.8.
M06.4.1-F4

Forwards, the tree multiplies along each path. Backwards, it keeps the leaves where ‘free’ was seen, adds them to get the evidence, and asks what share of them came through spam.

The leaves are the four joint probabilities. Two of them have ‘free’:

\[ \begin{aligned} &\Prob(\text{spam and free}) \\ &\quad= 0.4 \times 0.6 = 0.24, \\ &\Prob(\text{not spam and free}) \\ &\quad= 0.6 \times 0.1 = 0.06. \end{aligned} \]

Their sum is the evidence, \({\Prob(\text{free}) = 0.30}\), and the posterior is the spam share:

\[ \Prob(\text{spam} \mid \text{free}) = \frac{0.24}{0.30} = 0.8 . \]

Seeing ‘free’ has moved the probability of spam from the prior 0.4 up to 0.8.

The other answer. What if the email does not contain ‘free’? Now keep the other two leaves, 0.16 and 0.54, whose sum is 0.70:

\[ \begin{aligned} \Prob(\text{spam} \mid \text{no free}) &= \frac{0.16}{0.70} \\ &= \frac{8}{35} \approx 0.229 . \end{aligned} \]

Missing ‘free’ is weak evidence against spam, and the probability falls from 0.4 to about 0.23. The two posteriors average back to the prior, weighted by how often each case happens: \({0.30 \times 0.8} + {0.70 \times 8/35} = 0.24 + 0.16 = 0.4\). Before you open an email, you cannot expect the evidence to push your belief up or down on average.

Code

First Bayes’ rule as a function of the three numbers we are given, with exact fractions.

from fractions import Fraction as Fr

def bayes(prior, like_h, like_not_h):
    """P(H | E) from the prior P(H) and the likelihoods P(E | H) and P(E | not H)."""
    evidence = like_h * prior + like_not_h * (1 - prior)   # law of total probability
    return like_h * prior / evidence

prior = Fr(4, 10)
p_free = Fr(6, 10) * prior + Fr(1, 10) * (1 - prior)
print("P(free)           =", p_free)
print("P(spam | free)    =", bayes(prior, Fr(6, 10), Fr(1, 10)))
print("P(spam | no free) =", bayes(prior, Fr(4, 10), Fr(9, 10)))
P(free)           = 3/10
P(spam | free)    = 4/5
P(spam | no free) = 8/35

The no-‘free’ case passes the likelihoods of not seeing the word, \({1 - 0.6}\) and \({1 - 0.1}\). Now the same answer by counting, the way the grid did it: list the 100 emails and keep the ones with ‘free’.

emails = ([("spam", True)] * 24 + [("spam", False)] * 16
          + [("not spam", True)] * 6 + [("not spam", False)] * 54)
with_free = [label for label, free in emails if free]
print(len(with_free), "emails say 'free';", with_free.count("spam"), "of them are spam:",
      Fr(with_free.count("spam"), len(with_free)))
30 emails say 'free'; 24 of them are spam: 4/5

A simulation checks the rule without using it. Each simulated email is spam with probability 0.4, and then contains ‘free’ with probability 0.6 or 0.1. Among the simulated emails with ‘free’, we count the share that is spam.

import numpy as np

rng = np.random.default_rng(seed=4)
n = 200_000
spam = rng.random(n) < 0.4
free = np.where(spam, rng.random(n) < 0.6, rng.random(n) < 0.1)
estimate = spam[free].mean()
print(f"{free.sum()} simulated emails say 'free'; the share that is spam is {estimate:.3f}")
59559 simulated emails say 'free'; the share that is spam is 0.799

The share settles close to 0.8; another seed would give a slightly different third decimal. Finally, pandas builds the whole table of conditional shares at once. With normalize="columns" each column is divided by its own total, so the column for ‘free’ holds \({\Prob(\text{spam} \mid \text{free})}\).

import pandas as pd

table = pd.crosstab(pd.Series(spam, name="spam"), pd.Series(free, name="free"), normalize="columns")
print(table.round(3))
free   False  True 
spam               
False  0.771  0.201
True   0.229  0.799

Explore

This module’s interactive, the frequency grid, is in the next lesson. It counts Bayes’ rule out on 1,000 people, as the grid above counted it on 100 emails.

Pitfalls

Two panels showing the 30 emails that contain free as three rows of filled squares: 24 blue spam squares on the left and 6 sky-blue not-spam squares on the right. The left panel, marked with a cross and titled Only the spam branch, frames only the 24 spam squares; the 6 others are faded and labelled not spam, left out; under it, 0.24 over 0.24 equals 1, every free email is spam. The right panel, marked with a tick and titled Every way to see free, frames all 30 squares; under it, 0.24 over 0.24 plus 0.06 equals 0.8, 24 of the 30 are spam.
M06.4.1-F5

Wrong beside right. The evidence must count every way the evidence can happen. Leave out the not-spam branch and every ‘free’ email looks like spam.

“P(spam | free) and P(free | spam) are the same thing.”

They answer different questions and divide by different totals: 30 ‘free’ emails in one, 40 spam emails in the other. They differ by the factor \({\Prob(\text{spam}) / \Prob(\text{free})}\) and agree only when those two are equal. Mixing them up is so common that the next lesson gives it two names.

“The denominator is the likelihood of the hypothesis I care about.”

The evidence \(\Prob(E)\) counts every route to \(E\), through \(H\) and through \(H^c\). Dividing \({0.24}\) by \({0.24}\) alone gives 1, as in the left panel. The right denominator is \({0.24 + 0.06}\).

“The two likelihoods should add up to 1.”

\({\Prob(\text{free} \mid \text{spam}) = 0.6}\) and \({\Prob(\text{free} \mid \text{not spam}) = 0.1}\) add up to 0.7, and nothing is wrong. They are probabilities of the same event under two different conditions. What adds up to 1 is a pair with the same condition: the chances that a spam email does and does not say ‘free’ (0.6 and 0.4), or the two posteriors for an email that says ‘free’ (0.8 for spam and 0.2 for not spam). A naive Bayes classifier (ML02.6) stores one likelihood per word for each class, and these never need to add up to 1 across classes.

Check yourself

Recall

1. State Bayes’ rule and name its four parts.

Show answer

\({\Prob(H \mid E)} = {\Prob(E \mid H)\,\Prob(H) / \Prob(E)}\). \(\Prob(H)\) is the prior, \({\Prob(E \mid H)}\) the likelihood, \(\Prob(E)\) the evidence and \({\Prob(H \mid E)}\) the posterior.

2. Which single definition is used twice to derive Bayes’ rule?

Show answer

The definition of conditional probability, \({\Prob(A \mid B) = \Prob(A \cap B) / \Prob(B)}\): once with \(E\) as the condition, and once with \(H\) as the condition to write \({\Prob(H \cap E)}\) as \({\Prob(E \mid H)\,\Prob(H)}\).

3. How do you find the evidence \(\Prob(E)\) when it is not given?

Show answer

By the law of total probability: \(\Prob(E) = {\Prob(E \mid H)\,\Prob(H)} + {\Prob(E \mid H^c)\,\Prob(H^c)}\), the sum of the leaves of the tree where \(E\) happens.

4. In the inbox, \({\Prob(\text{free} \mid \text{spam}) = 0.6}\) but \({\Prob(\text{spam} \mid \text{free}) = 0.8}\). Why are they different?

Show answer

Both have the same overlap, 24 emails, but the first divides by the 40 spam emails and the second by the 30 emails with ‘free’.

5. If the evidence is impossible under a hypothesis, \({\Prob(E \mid H) = 0}\), what is the posterior?

Show answer

The posterior is 0. The numerator of Bayes’ rule is 0, so the evidence rules the hypothesis out (as long as \({\Prob(E) > 0}\)).

Exercises

1. ★ In the inbox, the word ‘winner’ is in 10 of the 40 spam emails and in 3 of the 60 others. Find \({\Prob(\text{spam} \mid \text{winner})}\) with Bayes’ rule, then check it by counting.

Show solution

The likelihoods are \({10/40 = 0.25}\) and \({3/60 = 0.05}\), and the prior is 0.4. The evidence is \({0.25 \times 0.4} + {0.05 \times 0.6} = 0.10 + 0.03 = 0.13\), so the posterior is \({0.10 / 0.13 = 10/13 \approx 0.769}\). Counting: 13 emails say ‘winner’ and 10 of them are spam, \({10/13}\) again.

2. ★★ A factory has two machines. Machine A makes 30% of the parts and 2% of its parts are faulty; machine B makes the other 70% and 5% of its parts are faulty. A part is found to be faulty. What is the probability that machine A made it? And if a part is not faulty?

Show solution

Let \(H\) be “made by A” and \(E\) be “faulty”. The leaves with a fault are \({0.3 \times 0.02 = 0.006}\) and \({0.7 \times 0.05 = 0.035}\), so \({\Prob(E) = 0.041}\) and

\[ \begin{aligned} \Prob(A \mid \text{faulty}) &= \frac{0.006}{0.041} \\ &= \frac{6}{41} \approx 0.146 . \end{aligned} \]

The leaves without a fault are \({0.3 \times 0.98 = 0.294}\) and \({0.7 \times 0.95 = 0.665}\), whose sum is 0.959, so \({\Prob(A \mid \text{not faulty})} = {0.294 / 0.959} = {42/137} \approx {0.307}\). A fault makes A less likely than its prior 0.3, because A makes fewer faults than B.

3. ★★★ Suppose \({0 < \Prob(H) < 1}\) and \({\Prob(E) > 0}\). Prove that the evidence raises the probability of the hypothesis, \({\Prob(H \mid E) > \Prob(H)}\), exactly when \({\Prob(E \mid H) > \Prob(E \mid H^c)}\).

Show solution

Divide Bayes’ rule by \({\Prob(H) > 0}\): \({\Prob(H \mid E) > \Prob(H)}\) holds exactly when \({\Prob(E \mid H) / \Prob(E) > 1}\), that is when \({\Prob(E \mid H) > \Prob(E)}\). Write \(p\) for \(\Prob(H)\), \(\ell_1\) for \({\Prob(E \mid H)}\) and \(\ell_0\) for \({\Prob(E \mid H^c)}\), and expand the evidence by the law of total probability:

\[ \begin{aligned} &\ell_1 > \ell_1 p + \ell_0 (1 - p) \\ &\iff \ell_1 (1 - p) > \ell_0 (1 - p) \\ &\iff \ell_1 > \ell_0 . \end{aligned} \]

The second line subtracts \({\ell_1 p}\) from both sides, and the third divides by \({1 - p > 0}\). So evidence supports a hypothesis exactly when it is more likely under the hypothesis than under its complement. Lesson M06.4.3 turns the ratio of these two likelihoods into the main character.

Summary card

Turning a conditional around keeps the overlap and swaps the total.

The anchor figure again: the grid of 100 emails framed two ways, the 40 spam emails with 24 that say free, giving 0.6, and the 30 free emails with 24 that are spam, giving 0.8.
M06.4.1-F1

Same overlap, different totals.

  • Bayes’ rule. \({\Prob(H \mid E)} = {\Prob(E \mid H)\,\Prob(H) / \Prob(E)}\).
  • The names. Prior \(\Prob(H)\), likelihood \({\Prob(E \mid H)}\), evidence \(\Prob(E)\), posterior \({\Prob(H \mid E)}\).
  • The evidence. \(\Prob(E) = {\Prob(E \mid H)\,\Prob(H)} + {\Prob(E \mid H^c)\,\Prob(H^c)}\).
  • The inbox. \({0.6 \times 0.4 / 0.30 = 0.8}\).

Sources and next steps

Sources. Each was opened and checked against this lesson on 14 November 2026.

  • Charles M. Grinstead and J. Laurie Snell, Introduction to Probability, second edition (free under the GNU FDL). Section 4.1 calls the events hypotheses and evidence, names the prior and posterior probabilities, derives Bayes’ formula from equations (4.1) to (4.3), and draws a forward and a reverse tree in Figure 4.5.
  • Hossein Pishro-Nik, Introduction to Probability, Statistics, and Random Processes, Section 1.4.3. Bayes’ rule for two events and for a partition, with the law of total probability in the denominator.
  • Jeremy Orloff and Jonathan Bloom, Reading 3: Conditional probability, independence and Bayes’ theorem, MIT OpenCourseWare 18.05, Spring 2022. Section 7 states Bayes’ theorem, proves it from the multiplication rule, and describes it as the way to invert a conditional probability.
  • pandas, pandas.crosstab. With normalize="columns" each column is divided by its own sum.

Next. M06.4.2 Base rates and frequency grids counts Bayes’ rule out on 1,000 people and shows how a good test for a rare condition still gives mostly false alarms.

This lesson unlocks

M06.4.2 Base rates and frequency grids

Your progress is saved in this browser only. No account, no tracking.
Back to top