\( \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\mat}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M06.8 · Common distributions and their stories

A free, illustrated module of 4 lessons and one interactive in M06 Probability I: Bernoulli and binomial: counting successes; Geometric and exponential: waiting for the first success; Poisson: counting rare events; A map of the families.

M06.8 L1 · Core 4 lessons · about 100 min Live · in review

Part of M06 Probability I in the Mathematics track.

A handful of distributions model most of the random numbers machine learning meets, and each one is a story: successes counted in n trials, a wait for the first success, rare events counted in a window, a point picked from an interval, a sum of many small effects, one label out of K. These four lessons derive the Bernoulli, binomial, geometric, exponential and Poisson distributions from their stories, with their means and variances, show how one family turns into another in a limit, and end with a map of every family of the course and a guide to choosing one for data.

What you need first. Random variables, PMFs and CDFs (M06.5), densities and the inverse CDF (M06.6), expectation, linearity and variance (M06.7), and counting with binomial coefficients (M06.2.3). The second and third lessons use the derivative of ln (M04.3.4), integration by parts (M04.7.2) and the geometric and exponential series (M04.8). The code uses NumPy and scipy.stats, explaining each call.

Lessons

1. Bernoulli and binomial: counting successes M06.8.1 25 min

   Run n independent trials, each a success with chance p, and count the successes. Derive the binomial PMF as a count of sequences times the chance of each, get its mean np and variance np(1 - p) from indicators, and score a classifier on 10 test examples.

2. Geometric and exponential: waiting for the first success M06.8.2 25 min

   How many trials until the first success, and how long until the next event? Derive the geometric PMF and its mean 1/p, pack ever more trials into each minute to reach the exponential wait e^(-lambda t), and prove that both forget how long you have waited.

3. Poisson: counting rare events M06.8.3 25 min

   Typos on a page, requests in a second: many chances, each unlikely. Derive the Poisson PMF e^(-lambda) lambda^k / k! as the limit of Bin(n, lambda/n), show that it sums to 1 and that its mean and variance are both lambda, and set Bin(100, 0.03) beside Pois(3).

4. A map of the families M06.8.4 25 min

   Bernoulli, binomial, geometric, Poisson, uniform, exponential, normal and categorical on one map, with arrows for how one becomes another. Choose a family for data by its story, see what goes wrong when the story does not fit, and meet the categorical distribution behind a softmax.

The interactive

M06.8-X1 · Distribution map. The families drawn as a map with arrows for how one becomes another. Choose a family to see its PMF or density with sliders for its parameters and its mean and standard deviation marked; choose an arrow and step n up to watch the limit happen, such as a binomial with np held fixed settling onto a Poisson. It lives in A map of the families.