\( \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\mat}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

Joint distributions as tables

Two random variables on the same outcomes have a joint PMF: one probability for every pair of values, laid out as a table. Build the course’s table from three coin flips, read any event about the pair as a sum of cells, and count the table in NumPy and pandas.

M06.9.1L1 Core25 min

In review

The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

Before this lesson: M06.5.4 Independent random variables and their sums M06.3.1 Conditional probability: shrinking the sample space

What you will be able to do

Define the joint PMF of two discrete random variables and build its table by listing outcomes.
Check that a table is a joint PMF: every cell non-negative and all cells adding to 1.
Find the probability of any event about the pair, such as X = Y, by adding the cells it contains.
Explain why the two separate PMFs of a dependent pair cannot answer questions about the pair.
Build a joint table from scratch with Counter and NumPy, and from data with pandas.crosstab.

The question

Flip a fair coin three times. Let \(X\) count the heads in flips 1 and 2, and \(Y\) the heads in flips 2 and 3. Each count is 0, 1 or 2 with chances \(1/4\), \(1/2\), \(1/4\). What is the chance that \({X = Y}\)? The two PMFs, however carefully drawn, cannot say. Machine learning meets the same problem with every data set: a feature and a label each have their own spread, but what a model learns is how they occur together.

Intuition

\(X\) and \(Y\) are two numbers read off the same three flips. So list the 8 equally likely outcomes, write the pair \({(x, y)}\) beside each, and sort them into a 3 by 3 grid with one cell for each possible pair. The share of outcomes that lands in a cell is the probability of that pair.

M06.9.1-F1

Sorting the 8 outcomes by their pair \({(x, y)}\). Each cell holds the outcomes that give its pair; two cells are empty and the middle cell holds two outcomes.

That grid is the joint distribution of \(X\) and \(Y\). It holds everything there is to know about the pair: the chance of each pair of values, and so the chance of any statement about both. Two cells are empty, because \(X\) and \(Y\) can never be 0 and 2 at once: that would need flip 2 to be tails for \(X\) and heads for \(Y\).

Build it up

Step 1. Two variables, one experiment. A random variable is a function of the outcome (M06.5.1). Here two functions read the same outcome, and they overlap: flip 2 counts in both.

M06.9.1-F2

\(X\) reads flips 1 and 2; \(Y\) reads flips 2 and 3. They share flip 2, so they are tied together.

Step 2. Name each pair as an event. The outcomes with \({X = x}\) and \({Y = y}\) form a set, an event. For example, \({X = 1}\) and \({Y = 1}\) is the set \(\{\mathrm{HTH}, \mathrm{THT}\}\), with probability \(2/8\). The comma is short for “and”, as in lesson M06.5.4.

Step 3. One cell per pair. Do this for all nine pairs \({(x, y)}\) with \(x\) and \(y\) in \(\{0, 1, 2\}\). Every outcome gives exactly one pair, so the nine events split the 8 outcomes between them with nothing left over and nothing counted twice. In eighths, the rows \({x = 0, 1, 2}\) read \({(1, 1, 0)}\), \({(1, 2, 1)}\) and \({(0, 1, 1)}\). That is the course’s joint table, which the rest of this module uses throughout.

Step 4. Ask about the pair. Now a question about both variables is a set of cells. “\({X = Y}\)” is the diagonal; “\({X > Y}\)” is the three cells below it. Add the cells and you have the answer.

The formal statement

Definition: joint PMF

Let \(X\) and \(Y\) be discrete random variables defined on the same sample space. Their joint PMF is

\[ p_{X,Y}(x, y) = \Prob(X = x, Y = y), \]

the probability that \({X = x}\) and \({Y = y}\) both happen, for every pair of values \({(x, y)}\). When no confusion can arise we write \(p(x, y)\).

The joint PMF has two properties, and each follows from the axioms (M06.1.3).

It is never negative. Each \(p(x, y)\) is the probability of an event, and probabilities are at least 0.

Its cells add to 1. Each outcome \(\omega\) gives one value \(X(\omega)\) and one value \(Y(\omega)\), so it lies in exactly one cell event. The cell events are therefore disjoint and together they cover \(\Omega\). Additivity gives

\[ \sum_{x} \sum_{y} p(x, y) = \Prob(\Omega) = 1. \]

The double sum runs over every row \(x\) and, inside it, every column \(y\): it adds the whole table.

Events about the pair. Let \(A\) be any set of pairs, such as all pairs with \({x = y}\). The event “\({(X, Y)}\) lies in \(A\)” is the disjoint union of the cells whose pair is in \(A\), so

\[ \Prob\big((X, Y) \in A\big) = \sum_{(x, y) \in A} p(x, y). \]

That one formula answers every question about the pair. On the course’s table:

\[ \begin{aligned} &\Prob(X = Y) \\ &= p(0,0) + p(1,1) + p(2,2) \\ &= \tfrac18 + \tfrac28 + \tfrac18 = \tfrac12. \end{aligned} \]

More than two variables. The same idea works for three or more variables: the joint PMF of \(X\), \(Y\) and \(Z\) gives \({\Prob(X = x, Y = y, Z = z)}\) and fills a cube instead of a table. Two variables are enough to see every idea of this module.

Go deeper: why the cells are a partition of the outcomes

Write \(C_{x,y}\) for the set of outcomes \(\omega\) with \({X(\omega) = x}\) and \({Y(\omega) = y}\). Two different cells cannot share an outcome: if \({(x, y) \neq (x', y')}\), an outcome in both would have two different values of \(X\) or of \(Y\), and a function has one value at each input. And every outcome lies in some cell, namely \(C_{X(\omega), Y(\omega)}\). So the cells, with the empty ones dropped, are a partition of \(\Omega\) in the sense of M06.3.3. When \(X\) and \(Y\) take only finitely many values the sum is finite; when they take infinitely many, the third axiom covers the countable sum.

Worked example

Find \({\Prob(X = Y)}\) and \({\Prob(X > Y)}\) for the course’s table.

Pick the cells. \({X = Y}\) holds the diagonal cells \({(0, 0)}\), \({(1, 1)}\) and \({(2, 2)}\). \({X > Y}\) holds the cells with a larger row than column: \({(1, 0)}\), \({(2, 0)}\) and \({(2, 1)}\).

M06.9.1-F3

An event about the pair is a set of cells. Add the yellow cells: \(1/2\) on the left, \(1/4\) on the right.

Add them.

\[ \begin{aligned} \Prob(X = Y) &= \tfrac18 + \tfrac28 + \tfrac18 = \tfrac48 = \tfrac12, \\ \Prob(X > Y) &= \tfrac18 + 0 + \tfrac18 = \tfrac28 = \tfrac14. \end{aligned} \]

Check by symmetry. Swapping flips 1 and 3 swaps \(X\) and \(Y\) and leaves the chances unchanged, so \({\Prob(X < Y) = \Prob(X > Y) = 1/4}\). The three events \({X < Y}\), \({X = Y}\) and \({X > Y}\) cover every outcome without overlap, and indeed \({1/4 + 1/2 + 1/4 = 1}\).

Check against the outcomes. \({X = Y}\) happens on HHH, HTH, THT and TTT, which is 4 of the 8 outcomes, and \({X > Y}\) on HHT and HTT: 2 of the 8. The table and the list agree.

Code

From scratch: list the outcomes, count each pair with Counter, and pour the counts into a 3 by 3 NumPy array whose row index is \(x\) and column index is \(y\).

import itertools
from collections import Counter
from fractions import Fraction

import numpy as np

def pair(word):
    """(X, Y) for three flips: heads in flips 1 and 2, heads in flips 2 and 3."""
    h = [1 if ch == "H" else 0 for ch in word]
    return h[0] + h[1], h[1] + h[2]

outcomes = ["".join(w) for w in itertools.product("HT", repeat=3)]
counts = Counter(pair(w) for w in outcomes)
joint = {(x, y): Fraction(counts[(x, y)], len(outcomes)) for x in range(3) for y in range(3)}

table = np.zeros((3, 3))
for (x, y), p in joint.items():
    table[x, y] = p                           # row x, column y
print("table in eighths:\n", table * 8)
print("total:", table.sum())

p_equal = sum(p for (x, y), p in joint.items() if x == y)
p_greater = sum(p for (x, y), p in joint.items() if x > y)
print("P(X = Y) =", p_equal, "  P(X > Y) =", p_greater)

table in eighths:
 [[1. 1. 0.]
 [1. 2. 1.]
 [0. 1. 1.]]
total: 1.0
P(X = Y) = 1/2   P(X > Y) = 1/4

With data instead of a list of outcomes, pandas.crosstab counts each pair of labels for you, and normalize=True divides by the number of rows. Here are 10,000 simulated triples of flips.

import pandas as pd

rng = np.random.default_rng(0)
h = rng.integers(0, 2, size=(10_000, 3))      # each row: three flips, 1 = heads
X = pd.Series(h[:, 0] + h[:, 1], name="X")
Y = pd.Series(h[:, 1] + h[:, 2], name="Y")
freq = pd.crosstab(X, Y, normalize=True)
print(freq.round(3))
print("simulated P(X = Y):", round(float((X == Y).mean()), 3))

Y      0      1      2
X                     
0  0.123  0.125  0.000
1  0.127  0.247  0.131
2  0.000  0.126  0.121
simulated P(X = Y): 0.491

M06.9.1-F5

The simulated table (left) and the exact one (right). The two impossible cells stay at exactly 0; the others land within a few thousandths of their eighths.

Every simulated cell sits within a hundredth of the exact value; the exact digits depend on the seed. The two cells that are impossible stay at 0 however long the simulation runs.

Explore

This module’s interactive, the joint table explorer, lives in lesson M06.9.3. It starts from this lesson’s table, so you can already use it to read off cells and check that they add to 1.

Pitfalls

M06.9.1-F4

Wrong beside right. Multiplying the two PMFs builds a table for an independent pair and gives \(3/8\); the real table gives \(1/2\).

“I know the PMF of \(X\) and the PMF of \(Y\), so I can find any probability about them.”

Not about both at once. Multiplying the PMFs assumes independence (M06.5.4) and gives \({\Prob(X = Y)} = {1/16 + 4/16 + 1/16} = {3/8}\), but the shared flip pushes \(X\) and \(Y\) to agree, and the truth is \(1/2\). In machine learning this is the gap between describing each column of a data set and describing the rows: two columns can each look ordinary while being tightly linked.

“Each row of the table should add to 1.”

Only the whole table adds to 1. The rows of the course’s table add to \(2/8\), \(4/8\) and \(2/8\): how much probability sits in each row is itself information (lesson M06.9.2). Rows that add to 1 are what you get after conditioning (lesson M06.9.3).

“An empty cell is a mistake in the table.”

A cell of 0 says that the pair is impossible, and that is often the most informative entry. Here \({p(0, 2) = 0}\) because \({X = 0}\) forces flip 2 to be tails, while \({Y = 2}\) forces it to be heads.

Check yourself

Recall

1. What does the joint PMF \(p(x, y)\) give?

Show answer

The probability that \({X = x}\) and \({Y = y}\) both happen, \({\Prob(X = x, Y = y)}\), for each pair of values.

2. Which two properties does every joint PMF have?

Show answer

Every cell is at least 0, and all the cells together add to 1.

3. How do you find the probability of an event about the pair, such as \({X = Y}\)?

Show answer

Add the cells whose pairs belong to the event: \({\Prob((X, Y) \in A)} = {\sum_{(x, y) \in A} p(x, y)}\).

4. In the course’s table, why is \({p(2, 0) = 0}\)?

Show answer

\({X = 2}\) needs flip 2 to be heads, and \({Y = 0}\) needs it to be tails. No outcome does both.

5. What does pandas.crosstab(X, Y, normalize=True) return?

Show answer

The table of relative frequencies: for each pair of labels, the share of the rows that have that pair. It estimates the joint PMF from data.

Exercises

1. ★ Using the course’s table, find \({\Prob(X + Y \geq 3)}\).

Show solution

The pairs with \({x + y \geq 3}\) are \({(1, 2)}\), \({(2, 1)}\) and \({(2, 2)}\), so the probability is \({1/8 + 1/8 + 1/8 = 3/8}\). Check with the outcomes: \({X + Y = H_1 + 2H_2 + H_3}\) is at least 3 on HHH, HHT and THH, which is 3 of the 8.

2. ★★ Roll two fair dice. Let \(M\) be the larger of the two faces and \(N\) the smaller (equal if the faces match). Find \({p(3, 1)}\) and \({p(3, 3)}\), and explain why \({p(1, 3) = 0}\).

Show solution

\({M = 3, N = 1}\) happens on the ordered pairs \({(3, 1)}\) and \({(1, 3)}\): two of the 36, so \({p(3, 1) = 2/36}\). \({M = 3, N = 3}\) happens only on \({(3, 3)}\), so \({p(3, 3) = 1/36}\). The larger face can never be smaller than the smaller one, so the pair \({M = 1, N = 3}\) is impossible and \({p(1, 3) = 0}\): every cell above the diagonal is empty.

3. ★★★ Keep three fair flips, but now let \(X\) be the heads in flips 1 and 2 and \(W\) the heads in flip 3 alone. Build the joint table of \(X\) and \(W\), and show that each cell is the product of \({\Prob(X = x)}\) and \({\Prob(W = w)}\). Why does that happen here but not for the course’s table?

Show solution

\(W\) is 0 or 1, each with probability \(1/2\). \({X = x, W = w}\) fixes flips 1 and 2 to give \(x\) heads and flip 3 to give \(w\). The number of outcomes is (ways to get \(x\) heads in two flips) times 1, which is 1, 2, 1 for \({x = 0, 1, 2}\), whatever \(w\) is. So the table in eighths has rows \({(1, 1)}\), \({(2, 2)}\), \({(1, 1)}\). Each cell equals \({p_X(x) \times \tfrac12}\): for example \({p(1, 0) = 2/8 = \tfrac12 \times \tfrac12}\). It works because \(X\) and \(W\) read different flips, so they are independent. The course’s \(X\) and \(Y\) share flip 2, and the products fail, for example \({p(0, 2) = 0}\) while \({\tfrac14 \times \tfrac14 = 1/16}\).

Summary card

A joint distribution is a table: one probability for every pair of values.

The anchor figure again: the eight outcomes of three flips with their pairs (x, y), sorted into the 3 by 3 joint table of X and Y with cells 1/8, 1/8, 0; 1/8, 2/8, 1/8; 0, 1/8, 1/8. — M06.9.1-F1
Eight outcomes sorted into nine cells.

Joint PMF. \({p(x, y) = \Prob(X = x, Y = y)}\).
Valid table. Every cell \({\geq 0}\); all cells add to 1.
Events. \({\Prob((X, Y) \in A)} = {\sum_{(x, y) \in A} p(x, y)}\).
Course’s table, in eighths. Rows \({(1, 1, 0)}\), \({(1, 2, 1)}\), \({(0, 1, 1)}\); \({\Prob(X = Y) = 1/2}\).

Sources and next steps

Sources. Each was opened and checked against this lesson on 14 November 2026.

Hossein Pishro-Nik, Introduction to Probability, Statistics, and Random Processes, Section 5.1.1. The definition of the joint PMF, \({P_{XY}(x, y) = P(X = x, Y = y)}\), and the probability of a set of pairs as the sum of the joint PMF over that set; Table 5.1 is a joint PMF laid out as a table.
MIT OpenCourseWare 18.05, Reading 7a, Joint Distributions, Independence, Spring 2022. Section 3.1 builds joint tables for two dice, and Section 3.3 (Example 3) finds the probability of an event as the sum of its shaded cells.
pandas reference, pandas.crosstab: computes a frequency table of two factors; normalize=True (or 'all') divides every count by the total.

Next. M06.9.2 Marginals: summing out a variable recovers each variable’s own PMF from the table, and shows why the reverse is impossible.

This lesson unlocks

M06.9.2 Marginals: summing out a variable

Your progress is saved in this browser only. No account, no tracking.