\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M04.3 · Differentiation rules and the chain rule

A free, illustrated module of 4 lessons and one interactive in M04 Calculus I: Single Variable: Powers, sums and constant multiples; Exponentials, sine and cosine; Products and quotients; The chain rule.

M04.3 L1 · Core 4 lessons · about 100 min Live · in review

Part of M04 Calculus I: Single Variable in the Mathematics track.

Finding every slope from a limit is slow. These four lessons prove a handful of rules once, from the definition, and then use them everywhere: powers, sums and constant multiples, which differentiate every polynomial; exponentials, sine and cosine; products and quotients, which give the slope of the sigmoid; and the chain rule, which multiplies the slopes along a chain of functions like gear ratios and is the seed of backpropagation.

What you need first. The derivative as the limit of a difference quotient, the tangent line and linear approximation, and central differences (M04.2); the limit laws and the limit of sin(h)/h (M04.1); the binomial theorem (M01.8); exponentials and logarithms (M01.4); and the angle-addition formulas (M01.5).

Lessons

1. Powers, sums and constant multiples M04.3.1 25 min

   Three rules, each proved once from the definition of the derivative, replace a fresh limit for every function: the power rule from the binomial expansion, and the rules for constant multiples and sums from the limit laws. Together they differentiate every polynomial, the running loss included.

2. Exponentials, sine and cosine M04.3.2 25 min

   Three more derivatives found straight from the definition. The difference quotient of an exponential factors into the function times its slope at 0, and the base whose slope at 0 is 1 is e, so e^x is its own derivative. Angle addition and two small-angle limits turn the slope of the sine into the cosine.

3. Products and quotients M04.3.3 25 min

   The slope of a product is not the product of the slopes. A rectangle whose two sides grow shows what it is, and the quotient rule follows from it. Worked on x e^x, x^2 sin x and tan x, and on the sigmoid, whose slope sigma(1 - sigma) is the one logistic regression trains with.

4. The chain rule M04.3.4 25 min

   The slope of a function of a function is the product of the slopes of its links, as gear ratios multiply. Derived from the linear approximation, the chain rule gives the derivative of ln x, the power rule for every exponent, the slope of a squared error and the backward pass at the heart of backpropagation.

The interactive

M04.3-X1 · Chain-rule gears. Choose an inner and an outer function and turn the input x. Three linked gears show x, u = g(x) and y = f(u) turning; the two local gear ratios are printed, and their product is compared with the change in y measured for a small nudge of x. It lives in The chain rule.