\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M04.7 · Integration techniques

A free, illustrated module of 4 lessons and one interactive in M04 Calculus I: Single Variable: Substitution: the chain rule in reverse; Integration by parts: the product rule in reverse; Improper integrals: areas that run to infinity; When no antiderivative exists: Simpson’s rule and quad.

M04.7 L1 · Core 4 lessons · about 100 min Live · in review

Part of M04 Calculus I: Single Variable in the Mathematics track.

The table of antiderivatives covers powers, exponentials and a few more; most integrals that machine learning meets are not in it. These four lessons add the two great methods for finding antiderivatives by hand, each a differentiation rule run backwards: substitution, which undoes the chain rule and turns the sigmoid into softplus, and integration by parts, which undoes the product rule. Then they let the range run to infinity, so that a probability density can have total area 1 and a mean, and finish with the integrals no formula can do, such as areas under the bell curve, computed by Simpson’s rule and by scipy.integrate.quad.

What you need first. The chain rule and the product rule (M04.3), integrals as areas and Riemann sums (M04.5), and both parts of the fundamental theorem with the table of antiderivatives (M04.6). The code uses Python functions and loops, NumPy and scipy.integrate, explaining each new call.

Lessons

1. Substitution: the chain rule in reverse M04.7.1 25 min

   The chain rule turns a composition into a product; substitution runs it backwards. Spot an inner function whose derivative is also present, rename it u, and the integral becomes one from the table. Worked on a polynomial, on the slope of the bell curve and on the sigmoid, whose integral is softplus.

2. Integration by parts: the product rule in reverse M04.7.2 25 min

   The product rule, integrated, trades one integral for another: the integral of u dv is uv minus the integral of v du. Learn how to choose u, and work x eˣ, ln x and x² e⁻ˣ, which needs two rounds, with the boundary term of the definite form.

3. Improper integrals: areas that run to infinity M04.7.3 25 min

   An area over an infinite range is the limit of areas over growing finite ranges. Find exactly when the area under 1/xᵖ is finite, see a spike with finite area, and check that a probability density has total area 1, with the mean of the exponential distribution found by parts.

4. When no antiderivative exists: Simpson’s rule and quad M04.7.4 25 min

   The bell curve has no antiderivative built from elementary functions, so its areas must be computed numerically. Derive Simpson’s rule from a parabola through three samples, see why it is exact for cubics and why its error falls sixteenfold when the strips double, then use simpson and quad.

The interactive

M04.7-X1 · Substitution viewer. Two panels show the same integral before and after the change of variable u = g(x). Drag the limits a and b in the x panel and watch the limits g(a) and g(b) and the matching slices, of width dx and du = g’(x) dx, move in the u panel while the two areas stay equal. It lives in Substitution: the chain rule in reverse.