\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

ML01.1 · What learning from data means

A free, illustrated module of 4 lessons and one interactive in ML01 Machine Learning Foundations: Rules or examples; Features, labels and kinds of learning; A model is a function with knobs; Learning is choosing the knobs.

ML01.1 L1 · Core 4 lessons · about 100 min Live · in review

Part of ML01 Machine Learning Foundations in the Machine Learning track.

A program can get its rule from a programmer or from examples. These four lessons build the second kind by hand, on a spam filter small enough to check with pencil and paper. You will see what a dataset is and which kinds of learning its columns allow, why a model is a whole family of rules picked out by a few adjustable numbers, and how a loss turns “find the best rule” into a search that a computer can run.

What you need first. School algebra: comparing numbers, averages and reading a formula (M01.1), the graph of a function (M01.3) and sums written with a sigma (M01.7). A little Python: functions, loops and lists (P01.4, P01.6). No machine learning vocabulary is assumed.

Lessons

1. Rules or examples ML01.1.1 25 min

   A spam filter can get its rule from a programmer’s guess or from messages that people have already labelled. Build both on eight examples, count their mistakes by hand and in code, and learn the three things a program needs before it can be said to learn.

2. Features, labels and kinds of learning ML01.1.2 25 min

   A dataset is a table: each row is an example, each column is a feature, and one column may hold the label. See how the label column separates classification from regression, and what unsupervised and reinforcement learning do without one.

3. A model is a function with knobs ML01.1.3 25 min

   A model is not one rule but a whole family of rules, picked out by a few adjustable numbers. Meet two families, the threshold rule and the straight line, and see why no setting of the knobs can rescue a family that is too simple for the data.

4. Learning is choosing the knobs ML01.1.4 25 min

   A loss gives every setting of a model’s parameters one score, and training is the search for the lowest score. Run the simplest search there is, a grid, by hand and in code, and see why it cannot scale to models with many parameters.

The interactive

ML01.1-X1 · Rule versus learner. Drag a cut along a number line of labelled messages to write a spam rule by hand, then let a learner choose the cut from the examples. Move, add or relabel a message and watch the learned cut follow while the hand-written one stays put. It lives in Rules or examples.