\( \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\mat}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M05.1 · Functions of several variables and level sets

A free, illustrated module of 4 lessons and one interactive in M05 Multivariable and Matrix Calculus: Many inputs, one or many outputs; Level sets: contour lines and level surfaces; Seeing a function of many inputs: slices along lines and planes; Limits and continuity in several variables.

M05.1 L2 · Intermediate 4 lessons · about 100 min Live · in review

Part of M05 Multivariable and Matrix Calculus in the Mathematics track.

A model’s loss takes every weight at once and returns one number; a layer takes one list of numbers and returns another. These four lessons name such functions by their inputs and outputs, draw a function of any number of inputs through its level sets and through its slices along lines and planes, the method behind the loss-landscape pictures of deep learning, and say exactly what a limit and continuity mean when a point can be approached from infinitely many directions.

What you need first. Functions of two variables, contour lines and slices (M04.10), limits and continuity in one variable (M04.1), vectors, the dot product and norms (M02.1 to M02.3), and matrices as maps (M02.4). The logistic model of ML01.6 helps in lesson 2 but is explained again. The code uses NumPy and a little SciPy, explaining each call.

Lessons

1. Many inputs, one or many outputs M05.1.1 25 min

   A loss takes every weight of a model at once and returns one number; softmax takes a list of scores and returns a list of probabilities. Learn to name the domain, codomain and output shape of any function of many inputs, and write it for a whole batch in NumPy.

2. Level sets: contour lines and level surfaces M05.1.2 25 min

   A function of any number of inputs can be seen through the sets where it takes one value: curves for two inputs, surfaces for three, hyperplanes for a linear model. A classifier’s decision boundary is one of them. Find level sets by hand and on a grid.

3. Seeing a function of many inputs: slices along lines and planes M05.1.3 25 min

   A function of a thousand inputs cannot be drawn, but its values along a line are an ordinary curve and its values on a plane are a contour map. Build such slices by hand, use them as deep learning’s loss-landscape pictures do, and see what they can hide.

4. Limits and continuity in several variables M05.1.4 25 min

   In the plane a point can be approached along infinitely many paths, and a function can head for different values along different ones. Define limits with a ball of inputs, prove one by a squeeze, disprove one with two paths, and see why the losses of machine learning are continuous.

The interactive

M05.1-X1 · Level-set explorer. Choose the running bowl, the saddle, the running loss or a logistic score and slide the level: its contour is highlighted among its neighbours. Drag a point and turn a line through it, and a side panel draws the one-variable slice of the function along that line, with its height at the point and the level marked where the line crosses the contour. It lives in Seeing a function of many inputs: slices along lines and planes.