\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M04.8 · Sequences, series and Taylor approximation

A free, illustrated module of 4 lessons and one interactive in M04 Calculus I: Single Variable: Sequences and the geometric series; Does the sum settle? Convergence tests; Taylor polynomials; Taylor series and the size of the error.

M04.8 L1 · Core 4 lessons · about 100 min Live · in review

Part of M04 Calculus I: Single Variable in the Mathematics track.

Gradient descent never lands on the bottom of a loss, yet it gets there; a reward discounted for ever still adds up to a finite return. These four lessons make “settling” exact: a sequence that converges, a series that is the limit of its partial sums, and the geometric series behind both. They then sort sums that settle from sums that do not, from the harmonic series to the ratio test, build the Taylor polynomials that stand in for a function near a point (the line and the parabola that gradient descent and Newton’s method put in place of a loss), and bound their error, which says where a Taylor series equals its function and why a central difference beats a forward one.

What you need first. Limits at infinity (M04.1.3), the derivative and its rules (M04.2, M04.3), gradient descent and Newton’s method (M04.4.4), the fundamental theorem (M04.6), integration by parts and improper integrals (M04.7.2, M04.7.3), and finite geometric sums (M01.7.3). The code uses NumPy and a little SciPy, explaining each call.

Lessons

1. Sequences and the geometric series M04.8.1 25 min

   A sequence converges when its terms settle on one value. Make that exact with a tolerance and a cut-off, prove that r to the n shrinks to 0 exactly when the size of r is below 1, and sum the geometric series behind gradient descent’s error and a discounted return.

2. Does the sum settle? Convergence tests M04.8.2 25 min

   Terms that shrink to 0 do not guarantee a finite sum: 1 + 1/2 + 1/3 + … grows for ever, like ln n. Compare sums with areas to settle every sum of 1/n^p, justify the ratio test by a geometric series, and see why a learning-rate schedule of 1/t works.

3. Taylor polynomials M04.8.3 25 min

   The tangent line matches a curve’s height and slope; a polynomial of degree n can match its first n derivatives too. Derive the coefficients f^(k)(a)/k!, build e^x, sin x and cos x term by term, and see the line and the parabola that gradient descent and Newton’s method put in place of a loss.

4. Taylor series and the size of the error M04.8.4 25 min

   How far off can a Taylor polynomial be? Derive the remainder by integrating by parts, bound it by M|x - a|^(n+1)/(n+1)!, guarantee sin(0.5) and e, find where Taylor series converge, and see why a central difference beats a forward one.

The interactive

M04.8-X1 · Taylor builder. Choose e^x, sin x, cos x, ln(1 + x) or 1/(1 - x), drag the centre and step the degree from 0 to 12. Each Taylor polynomial is drawn over the function with the gap shaded, and the error at a movable probe is printed. For ln(1 + x) and 1/(1 - x) the band where the series converges is marked, and outside it more terms make the polynomials worse. It lives in Taylor polynomials.