Before this lesson
M01.7.1 Sequences: lists with a rule M01.2.3 Distance, from the plane to many dimensions
After this lesson you can
In review. The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.
Open almost any machine-learning text and one of the first formulas is the loss of a model. For a model that predicts \(\hat{y}_i\) where the target is \(y_i\), on \(n\) examples, it is written
\[ \closs{L} = \frac{1}{n} \sum_{i=1}^{n} \big(\cmodel{\hat{y}_i} - \ctruth{y_i}\big)^2 . \]
One Greek letter stands in for a whole loop over the data. What exactly does it tell you to do, and how can you work with it without writing out every term?
From the last lesson
1. What is the 20th term of 2, 5, 8, 11, and so on?
\({a_{20} = 2 + 19 \times 3 = 59}\).
2. Which term of the doubling sequence 1, 2, 4, 8 is 1024?
The 11th. The terms are \({a_n = 2^{\,n-1}}\) and \({1024 = 2^{10}}\), so \({n - 1 = 10}\).
The large letter \(\sum\) is the Greek capital sigma, the Greek S, for “sum”. It is an instruction to add things up, and the small writing around it says exactly what to add. Below it sits a counter and where it starts. Above it sits where the counter stops. To its right sits the thing to add for each value of the counter. The figure names each part and runs the loop it stands for, on the course’s four inputs \(x_1\) to \(x_4\), which are 1, 2, 3 and 4.
A sigma is a loop. The total starts at 0. The counter \(i\) takes the values 1, 2, 3, 4 in turn, and for each value the term \(x_i\) is added on. When the counter passes the upper limit, the loop stops and the total is the value of the sum.
Read the expression aloud as “the sum of \(x_i\) for \(i\) from 1 to 4”. In Python it is a loop that keeps a running total, and the code section writes exactly that loop.
Step 1. The parts. In
\[ \sum_{i=1}^{4} x_i \]
the letter \(i\) is the index, a counter. The lower limit 1 is where it starts, the upper limit 4 is where it stops, and \(x_i\) is the term: what gets added for each value of \(i\). The limits are both included, so a sum from 1 to 4 has 4 terms and a sum from 3 to 7 has \({7 - 3 + 1 = 5}\) terms.
Step 2. Writing a sum out. Put each value of the index into the term in turn, and add:
\[ \begin{aligned} \sum_{i=1}^{4} x_i &= x_1 + x_2 + x_3 + x_4 \\ &= 1 + 2 + 3 + 4 = 10 . \end{aligned} \]
The term can be any expression in the index. With \(k^2\) as the term:
\[ \sum_{k=1}^{5} k^2 = 1 + 4 + 9 + 16 + 25 = 55 . \]
Step 3. Folding a sum back. Going the other way, look for what changes from term to term. In \({3 + 6 + 9 + \dots + 30}\) the \(k\)-th term is \(3k\), and the last one, 30, is \({3 \times 10}\), so
\[ 3 + 6 + 9 + \dots + 30 = \sum_{k=1}^{10} 3k . \]
Step 4. The index’s name does not matter. The sums \({\sum_{i=1}^{4} x_i}\) and \({\sum_{j=1}^{4} x_j}\) give the same instructions and the same total. The index is a placeholder, like the name of the loop variable in code.
Step 5. Shifting the index. The doubling numbers \({1 + 2 + 4 + 8 + 16}\) can be counted from 0, with term \(2^k\), or from 1, with term \({2^{\,n-1}}\).
One sum, two labellings. Setting \({n = k + 1}\) moves both limits up by 1 and replaces \(k\) by \({n - 1}\) in the term. The five terms, and their total 31, do not change.
So shifting is safe as long as you change the limits and the term together.
Three rules make sums easy to handle. Each is derived by writing out the terms, using only the laws of arithmetic from M01.1.2.
Rule 1: a constant factor comes outside. For a number \(c\) that is the same in every term,
\[ \sum_{i=1}^{n} c\, a_i = c \sum_{i=1}^{n} a_i . \]
Why. Write out the left side. Every term has the factor \(c\), so the distributive law takes it out of the whole row at once:
\[ \begin{aligned} &c\,a_1 + c\,a_2 + \dots + c\,a_n \\ &\quad = c\,(a_1 + a_2 + \dots + a_n) . \end{aligned} \]
Rule 2: a sum of sums splits.
\[ \sum_{i=1}^{n} (a_i + b_i) = \sum_{i=1}^{n} a_i + \sum_{i=1}^{n} b_i . \]
Why. Write out the left side, take off the brackets, and move every \(b\) to the back. Adding in a different order or grouping gives the same total (the commutative and associative laws):
\[ \begin{aligned} &(a_1 + b_1) + \dots + (a_n + b_n) \\ &\quad = (a_1 + \dots + a_n) + (b_1 + \dots + b_n) . \end{aligned} \]
Rule 3: adding a constant \(n\) times gives \(n\) times it.
\[ \sum_{i=1}^{n} c = n\, c . \]
Why. The term has no \(i\) in it, so every one of the \(n\) terms is \(c\), and \({c + c + \dots + c}\) with \(n\) copies is \({n\,c}\).
Together. Rules 1 and 2, with \({c = -1}\), also split a difference:
\[ \sum_{i=1}^{n} (a_i - b_i) = \sum_{i=1}^{n} a_i - \sum_{i=1}^{n} b_i . \]
There is no such rule for products or squares. The pitfalls below show why.
The course’s data are the four points with \({x = 1, 2, 3, 4}\) and \({y = 2, 3, 5, 6}\), and the line \({\hat{y} = 1.4x + 0.5}\), taken as given here (ML01.4 derives it).
The mean. The mean of \(x\) is the sum divided by how many there are:
\[ \begin{aligned} \bar{x} &= \frac{1}{4} \sum_{i=1}^{4} x_i \\ &= \frac{1 + 2 + 3 + 4}{4} = \frac{10}{4} = 2.5 . \end{aligned} \]
In the same way \({\bar{y} = (2 + 3 + 5 + 6)/4 = 4}\), so the mean point is \((2.5, 4)\), as in M01.2.1.
The mean squared error. The line’s predictions are \({\hat{y}_i = 1.4x_i + 0.5}\), which gives 1.9, 3.3, 4.7 and 6.1. Each term of the loss is one error, squared.
The sigma, one term per point. Each error is a prediction minus a target. Squared, the four errors are 0.01, 0.09, 0.09 and 0.01. They add to 0.2, and dividing by 4 gives 0.05.
Written out, the sigma is
\[ \begin{aligned} L &= \tfrac14 \big[(-0.1)^2 + 0.3^2 \\ &\qquad {} + (-0.3)^2 + 0.1^2\big] \\ &= \tfrac14 \,(0.01 + 0.09 + 0.09 + 0.01) \\ &= \tfrac14 \times 0.2 = 0.05 . \end{aligned} \]
A distance. In M01.2.3 the distance between two points with \(n\) coordinates was written with dots. With a sigma it fits on one line:
\[ d(P, Q) = \sqrt{\sum_{i=1}^{n} (q_i - p_i)^2} . \]
For \({P = (1, 2)}\) and \({Q = (4, 6)}\) the sum is \({3^2 + 4^2 = 25}\), and \({\sqrt{25} = 5}\). In space, from \((1, 2, 3)\) to \((2, 4, 5)\), the sum is \({1 + 4 + 4 = 9}\), and \({\sqrt{9} = 3}\). The square root sits outside the sigma: add every squared change first, then take one root.
A sigma is a loop that keeps a running total. Here it is for the sum and the mean of \(x\).
# the inputs x_1 to x_4
x = [1, 2, 3, 4]
# the targets y_1 to y_4
y = [2, 3, 5, 6]
# the running total
total = 0
# i takes the values 0, 1, 2, 3
for i in range(len(x)):
# add the term for i
total = total + x[i]
print("sum of x:", total)
print("mean of x:", total / len(x))sum of x: 10
mean of x: 2.5Python’s index runs from 0 to 3, so x[i] covers \(x_1\) to \(x_4\) of the maths. The mean squared error is the same loop with a different term.
# the course's line
w, b = 1.4, 0.5
# running total of squares
squares = 0
for i in range(len(x)):
# prediction for example i
y_hat = w * x[i] + b
squares = squares + (y_hat - y[i]) ** 2
mse = squares / len(x)
print("sum of squared errors:", round(squares, 4))
print("mean squared error:", round(mse, 4))sum of squared errors: 0.2
mean squared error: 0.05The total prints as 0.2 after rounding. Unrounded it is a hair off, because a computer stores a number such as 1.4 only approximately (M01.1.1). The distance is one more loop, then a square root.
import math
P, Q = (1, 2), (4, 6)
squares = 0
# one pass per coordinate
for i in range(len(P)):
squares = squares + (Q[i] - P[i]) ** 2
print("by the sigma:", math.sqrt(squares))
print("by math.dist:", math.dist(P, Q))by the sigma: 5.0
by math.dist: 5.0Now the library versions. Python’s sum adds a list for you, and NumPy adds a whole array, or averages it, in one call. The last line is the hook’s formula: the loss in a single line.
import numpy as np
print("sum:", sum(x), " mean:", sum(x) / len(x))
# NumPy arrays
xs, ys = np.array(x), np.array(y)
print("numpy.sum:", np.sum(xs))
print("numpy.mean:", np.mean(xs), np.mean(ys))
print("loss:", round(np.mean((w * xs + b - ys) ** 2), 4))sum: 10 mean: 2.5
numpy.sum: 10
numpy.mean: 2.5 4.0
loss: 0.05This module’s interactive, the sum builder, lives in lesson M01.7.3. It writes every sum it builds three ways at once: as a sigma, as its terms written out, and as one number.
Wrong beside right, when the goal is \(\sum x_i^2\). Squaring the total counts the whole 10 by 10 square, 100. Adding the squares counts only the four squares on the diagonal, 30.
“The sum of the squares is the square of the sum.”
For \({x = 1, 2, 3, 4}\), the sum of the squares is \({1 + 4 + 9 + 16 = 30}\), but the square of the sum is \({10^2 = 100}\). Squaring the total also counts every product of two different terms, the off-diagonal blocks of the figure, which add to 70. Formulas for spread, such as the variance of S01.3, depend on keeping these two apart.
“A sum of products is the product of the sums.”
For the course’s data,
\[ \begin{aligned} \sum x_i y_i &= 2 + 6 + 15 + 24 \\ &= 47 , \end{aligned} \]
while
\[ \begin{aligned} &\Big(\sum x_i\Big)\Big(\sum y_i\Big) \\ &\quad = 10 \times 16 = 160 . \end{aligned} \]
Rule 1 moves a factor outside only when it is the same in every term, and \(y_i\) changes with \(i\).
“Adding a constant gives the constant.”
A term with no index in it is still added once for each value of the index. So \({\sum_{i=1}^{4} 0.5 = 4 \times 0.5 = 2}\), not 0.5.
“Python’s range(1, n) runs up to n.”
It stops one short. range(1, 4) gives 1, 2, 3, while the sigma from 1 to 4 includes 4. To run over positions 1 to \(n\), write range(1, n + 1), or loop over indices 0 to \({n - 1}\) with range(n) and remember that index \(i\) holds term \({i + 1}\).
1. Read \({\sum_{i=1}^{4} x_i}\) aloud, and name its four parts.
“The sum of \(x_i\) for \(i\) from 1 to 4.” The index is \(i\), the lower limit 1, the upper limit 4, and the term \(x_i\).
2. How many terms does \({\sum_{k=3}^{7} a_k}\) have?
Five: \({7 - 3 + 1 = 5}\). They are \(a_3, a_4, a_5, a_6\) and \(a_7\).
3. Write \({1 + 4 + 9 + 16 + 25}\) with a sigma.
\({\sum_{k=1}^{5} k^2}\), which is 55.
4. What is \({\sum_{i=1}^{10} 3}\)?
\({10 \times 3 = 30}\): the term 3 is added once for each of the ten values of \(i\).
5. Write the mean squared error of predictions \(\hat{y}_i\) against targets \(y_i\), over \(n\) examples, with a sigma.
\[ L = \frac{1}{n} \sum_{i=1}^{n} (\hat{y}_i - y_i)^2 . \]
1. ★ Write out and evaluate \({\sum_{k=1}^{4} (2k + 1)}\). Then get the same answer from the three rules.
Written out: \({3 + 5 + 7 + 9 = 24}\). By the rules, split the sum (rule 2), take out the 2 (rule 1) and add the constant 1 four times (rule 3):
\[ \begin{aligned} \sum_{k=1}^{4} (2k + 1) &= 2 \sum_{k=1}^{4} k + \sum_{k=1}^{4} 1 \\ &= 2 \times 10 + 4 = 24 . \end{aligned} \]
2. ★★ For the course’s inputs, \({\sum x_i = 10}\) and \({\sum x_i^2 = 30}\), with \(i\) from 1 to 4. Use only these two totals and the rules to find \({\sum_{i=1}^{4} (x_i - 2.5)^2}\), then check it term by term.
Expand the square:
\[ (x_i - 2.5)^2 = x_i^2 - 5x_i + 6.25 . \]
Then split the sum and use the rules:
\[ \begin{aligned} &\sum_{i=1}^{4} x_i^2 - 5 \sum_{i=1}^{4} x_i + \sum_{i=1}^{4} 6.25 \\ &\quad = 30 - 5 \times 10 + 4 \times 6.25 = 5 . \end{aligned} \]
Term by term, the four squares 2.25, 0.25, 0.25 and 2.25 add to 5. This is the total squared distance of the inputs from their mean, the start of the variance.
3. ★★★ Without using the four errors themselves, show that the errors \({\hat{y}_i - y_i}\) of the line \({\hat{y} = 1.4x + 0.5}\) on the course’s data add to zero. Then say why the loss squares the errors before adding them.
Split the sum (rule 2, for a difference), then split \({\hat{y}_i = 1.4x_i + 0.5}\) the same way and use rules 1 and 3:
\[ \begin{aligned} &\sum_{i=1}^{4} (\hat{y}_i - y_i) \\ &\quad = \sum_{i=1}^{4} \hat{y}_i - \sum_{i=1}^{4} y_i \\ &\quad = 1.4 \sum_{i=1}^{4} x_i + 4 \times 0.5 - 16 \\ &\quad = 14 + 2 - 16 = 0 . \end{aligned} \]
The errors \({-0.1, 0.3, -0.3, 0.1}\) do add to 0. A loss that added raw errors would call this line perfect, and would also call perfect a line with errors of \({-5}\) and \({+5}\). Squaring makes every error count as positive, so errors cannot cancel.
A sigma is a loop that adds: a counter, its start and stop, and a term.
Start at 0; add one term per value of the index.
Sources. Each was opened and checked against this lesson on 12 November 2026.
Next. M01.7.3 Arithmetic and geometric sums adds whole sequences in one line, and watches a geometric total level off.
This lesson unlocks
M01.7.3 Arithmetic and geometric sums ML01.1.4 Learning is choosing the knobs