Skip to main content

Length, distance and norms

The length of a vector is Pythagoras written with a dot product, and the distance between two points is the length of their difference. Learn to normalise a vector, and meet two other rulers, the 1-norm and the max-norm, through their unit shapes.
M02.3.2L1 Core25 minIn review

Before this lesson

M02.3.1 The dot product, two ways M02.1.2 Vectors as lists of numbers M02.1.4 Data as vectors

After this lesson you can

  • Compute the length of a vector with any number of entries as the square root of its dot product with itself.
  • Compute the distance between two points as the length of their difference.
  • Normalise a non-zero vector to a unit vector and explain why the result has length one.
  • Compute the 1-norm, the 2-norm and the max-norm of a vector and sketch the unit shape of each in the plane.

In review. The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

The question

A nearest-neighbour classifier labels a new point by finding the training points closest to it. A regulariser tells a model to keep its weights small. Both need a ruler for lists of numbers. What is the length of a vector with three, or three hundred, entries? And is there only one sensible ruler?

From the last lesson

1. What is \(\vect{a} \cdot \vect{a}\)?

Show answer

The length of \(\vect{a}\), squared: \(\norm{\vect{a}}^2 = a_1^2 + \cdots + a_n^2\).

2. What does the sign of \(\vect{a} \cdot \vect{b}\) tell you about the angle between the arrows?

Show answer

Positive means acute, zero means a right angle, negative means obtuse.

Intuition

You have used this ruler before. Lesson M02.1.2 found the length of an arrow with Pythagoras: the arrow is the long side of a right triangle and its coordinates are the short sides. So the arrow \((3, 4)\) has length \(\sqrt{3^2 + 4^2} = \sqrt{25} = 5\). Lesson M02.1.4 measured the distance between two data points as the length of their difference. The figure puts the two side by side.

Two panels on a grid. Left panel, titled Length of an arrow: a blue arrow v equals (3, 4) from the origin, drawn as the long side of a right triangle whose horizontal side is labelled 3 and whose vertical side is labelled 4, with a right-angle mark in the corner. The arrow is labelled 5, and underneath: the length of v equals the square root of 3 squared plus 4 squared, equals 5. Right panel, titled Distance between two points: two blue dots, p equals (1, 1) and q equals (4, 5), joined by a blue arrow labelled q minus p and 5. Under the arrow is the same right triangle with sides 3 and 4. Underneath: the length of q minus p equals the length of (3, 4), equals 5.
M02.3.2-F1

A reminder: length and distance use one ruler. Left: the length of an arrow is the long side of the right triangle made by its coordinates. Right: the distance between two points is the length of the arrow from one to the other.

In the right panel the arrow from \(\vect{p}\) to \(\vect{q}\) is the difference \(\vect{q} - \vect{p} = (4 - 1,\; 5 - 1) = (3, 4)\): the same triangle, moved. So the distance is 5.

We write \(\norm{\vect{v}}\) for the length of \(\vect{v}\), as before, and from now on we also call it the norm of \(\vect{v}\). This lesson does three new things with it. It writes the ruler with a dot product, which makes its properties short to prove. It scales vectors to length 1. And it meets two other rulers, which measure the same vector differently.

Build it up

Step 1. Recap: in the plane. \(\norm{(x, y)} = \sqrt{x^2 + y^2}\), by Pythagoras.

Step 2. Recap: in space, Pythagoras twice. Lesson M02.1.2 proved that the length of \((v_1, v_2, v_3)\) is \(\sqrt{v_1^2 + v_2^2 + v_3^2}\). Here is its argument once more, on \(\vect{v} = (1, 2, 2)\), the diagonal of a box 1 wide, 2 deep and 2 high.

Two drawings of the same box, 1 by 2 by 2, with a blue arrow v along its diagonal from one bottom corner to the opposite top corner. Left drawing, Step 1: across the floor: a yellow right triangle on the floor of the box has short sides 1 and 2 and long side root 5; underneath, floor diagonal equals the square root of 1 squared plus 2 squared, equals root 5. Right drawing, Step 2: up to the corner: a yellow right triangle stands upright on the floor diagonal, with short sides root 5 and 2 and the arrow v as its long side, labelled 3; underneath, the length of v equals the square root of 5 plus 2 squared, equals root 9, equals 3.
M02.3.2-F2

A reminder: length in space is Pythagoras used twice. First across the floor, then up the wall. The square roots undo each other in the middle, which leaves a plain sum of three squares.

The floor diagonal has length \(\sqrt{1^2 + 2^2} = \sqrt{5}\). It meets the height 2 at a right angle, so Pythagoras applies again:

\[ \begin{aligned} \norm{\vect{v}} &= \sqrt{\big(\sqrt{5}\big)^2 + 2^2} \\ &= \sqrt{1^2 + 2^2 + 2^2} = \sqrt{9} = 3 . \end{aligned} \]

Step 3. Recap: with any number of entries. A list of four or more numbers has no arrow to measure. The same lesson therefore took the recipe itself as the meaning of length: square every entry, add, take the square root.

Step 4. Say it with a dot product. “Square every entry and add” is the dot product of a vector with itself, rule 4 of the last lesson. So \(\norm{\vect{v}} = \sqrt{\vect{v} \cdot \vect{v}}\).

Step 5. Keep the direction, drop the length. A unit vector is a vector of length 1. Divide any non-zero vector by its own length and you get the unit vector that points the same way. This is called normalising. For \(\vect{v} = (3, 4)\) it gives \(\tfrac{1}{5}(3, 4) = (0.6, 0.8)\). We write it \(\hat{\vect{v}}\).

Step 6. Other rulers. The length above is the straight-line ruler. A taxi in a city of square blocks cannot drive along the diagonal. To reach \((3, 4)\) it must cover \(3 + 4 = 7\) blocks. A third ruler looks only at the largest entry: for \((3, 4)\) it reads 4. Each ruler is called a norm. To see how a ruler behaves, draw every vector that it measures as exactly 1. That set is its unit shape.

Three panels, each showing a blue shape centred on the origin with the points 1 marked on both axes and one black arrow from the origin to a point on the edge of the shape. First panel, 1-norm: a diamond with corners on the axes; the arrow ends at (0.5, 0.5); underneath, absolute value of x plus absolute value of y equals 1, and 0.5 plus 0.5 equals 1. Second panel, 2-norm: a circle; the arrow ends at (0.6, 0.8); underneath, square root of x squared plus y squared equals 1, and square root of 0.36 plus 0.64 equals 1. Third panel, max-norm: a square with sides through the points 1 on the axes; the arrow ends at (0.5, 1); underneath, the larger of absolute value of x and absolute value of y equals 1, and max of 0.5 and 1 equals 1.
M02.3.2-F3

Three rulers and their unit shapes: every vector that the ruler measures as exactly 1. The taxi ruler gives a diamond, the straight-line ruler a circle, and the largest-entry ruler a square.

The formal statement

NoteDefinition: norm, distance, unit vector

The Euclidean norm (or length, or 2-norm) of a vector \(\vect{v} = (v_1, \dots, v_n)\) in \(\R^n\) is

\[ \norm{\vect{v}} \;=\; \sqrt{\vect{v} \cdot \vect{v}} \;=\; \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2} . \]

The distance between two points \(\vect{p}\) and \(\vect{q}\) is \(\norm{\vect{q} - \vect{p}}\). A unit vector is a vector of norm 1.

The sum of squares is the length of lesson M02.1.2, and the distance is that of lesson M02.1.4. The new part is the middle: a norm is the square root of a dot product, so the rules of the dot product can now do the proving.

Three properties make this a good ruler.

Property 1: lengths are positive. \(\norm{\vect{v}} \ge 0\), and \(\norm{\vect{v}} = 0\) only for the zero vector. This is rule 4 of the dot product: a sum of squares is never negative, and it is zero only when every entry is zero.

Property 2: scaling a vector scales its length. For every number \(k\), \(\norm{k\vect{v}} = \abs{k}\,\norm{\vect{v}}\). Lesson M02.1.3 showed this from the coordinates. With the dot product it takes one line, in any dimension. Work with the square first:

\[ \begin{aligned} \norm{k\vect{v}}^2 &= (k\vect{v}) \cdot (k\vect{v}) \\ &= k^2\,(\vect{v} \cdot \vect{v}) = k^2\,\norm{\vect{v}}^2 . \end{aligned} \]

The second line pulls \(k\) out of each slot (rule 3, twice). Now take square roots. The square root of \(k^2\) is \(\abs{k}\), not \(k\): stretching by \(-2\) doubles a length, it does not make it negative.

Property 3: the triangle inequality. \(\norm{\vect{a} + \vect{b}} \le \norm{\vect{a}} + \norm{\vect{b}}\). Walk along \(\vect{a}\) and then along \(\vect{b}\), and you arrive at \(\vect{a} + \vect{b}\). The inequality says that this detour is never shorter than going straight. For \(\vect{a} = (3, 0)\) and \(\vect{b} = (0, 4)\) it reads \(5 \le 3 + 4\). The proof needs one more tool, the Cauchy-Schwarz inequality, and comes in the next lesson.

Three consequences follow at once.

Normalising works. Let \(\vect{v} \ne \vect{0}\) and put \(\hat{\vect{v}} = \vect{v} / \norm{\vect{v}}\). By property 2 with the positive number \(k = 1/\norm{\vect{v}}\),

\[ \norm{\hat{\vect{v}}} \;=\; \frac{1}{\norm{\vect{v}}}\,\norm{\vect{v}} \;=\; 1 . \]

Scaling by a positive number does not turn an arrow, so \(\hat{\vect{v}}\) points the same way as \(\vect{v}\). The zero vector cannot be normalised: it has no direction, and the formula would divide by zero.

Distance does not care about order. Lesson M02.1.4 checked this entry by entry. Property 2 gives it at once: \(\vect{p} - \vect{q} = (-1)(\vect{q} - \vect{p})\), so \(\norm{\vect{p} - \vect{q}} = \abs{-1}\,\norm{\vect{q} - \vect{p}} = \norm{\vect{q} - \vect{p}}\).

A stop on the way never shortens a trip. For three points, put \(\vect{a} = \vect{q} - \vect{p}\) and \(\vect{b} = \vect{r} - \vect{q}\). Then \(\vect{a} + \vect{b} = \vect{r} - \vect{p}\), and property 3 says \(\norm{\vect{r} - \vect{p}} \le \norm{\vect{q} - \vect{p}} + \norm{\vect{r} - \vect{q}}\).

NoteDefinition: 1-norm and max-norm

\[ \begin{aligned} \norm{\vect{v}}_1 &= \abs{v_1} + \abs{v_2} + \cdots + \abs{v_n}, \\ \norm{\vect{v}}_\infty &= \max\big(\abs{v_1}, \dots, \abs{v_n}\big) . \end{aligned} \]

When several rulers are in play, the Euclidean norm is written \(\norm{\vect{v}}_2\). The 1-norm adds up the sizes of the entries: the taxi’s distance. The max-norm (or infinity-norm) reports the size of the largest entry. Both have properties 1 to 3 as well, which is what earns them the name norm. For every vector the three are ordered,

\[ \norm{\vect{v}}_\infty \;\le\; \norm{\vect{v}}_2 \;\le\; \norm{\vect{v}}_1 , \]

which exercise 3 asks you to prove. The unit shapes show the same order from the other side. A ruler that gives bigger readings reaches 1 sooner, so its unit shape is smaller: the diamond fits inside the circle, and the circle fits inside the square.

Go deeper: why the 1-norm and the max-norm obey the same three properties

Property 1. Absolute values are never negative. A sum of them, or the largest of them, is zero only when every entry is zero.

Property 2. Each entry of \(k\vect{v}\) has size \(\abs{k v_i} = \abs{k}\,\abs{v_i}\). The common factor \(\abs{k}\) comes out of the sum, and out of the maximum.

Property 3. For single numbers, \(\abs{a_i + b_i} \le \abs{a_i} + \abs{b_i}\). Add these up over \(i\) to get \(\norm{\vect{a} + \vect{b}}_1 \le \norm{\vect{a}}_1 + \norm{\vect{b}}_1\). For the max-norm, each \(\abs{a_i} \le \norm{\vect{a}}_\infty\) and each \(\abs{b_i} \le \norm{\vect{b}}_\infty\), so every entry satisfies \(\abs{a_i + b_i} \le \norm{\vect{a}}_\infty + \norm{\vect{b}}_\infty\). In particular the largest one does.

Frontier note: two rulers, two regularisers

Regularisation adds the size of the weight vector to the loss, so that training prefers small weights. The ruler matters. Ridge regression penalises the squared 2-norm, \(\norm{\vect{w}}_2^2\), which shrinks all the weights. The lasso penalises the 1-norm, \(\norm{\vect{w}}_1\), and can set some weights exactly to zero, which removes those features from the model. The scikit-learn user guide describes both in these terms. Notice where the corners of the diamond sit: on the axes, at points with a coordinate that is exactly zero.

Worked example

Find the length of \(\vect{v} = (3, 4)\) and the unit vector along it.

A flow of five boxes. Top row, headed Find the length, in blue: v equals (3, 4); an arrow labelled square, add leads to 3 squared plus 4 squared equals 25; an arrow labelled square root leads to the length of v equals root 25 equals 5. From that box a vermillion arrow labelled divide v by its length leads down to a vermillion box, v-hat equals (0.6, 0.8). An arrow labelled check leads left to a yellow box: 0.6 squared plus 0.8 squared equals 1. The bottom row is headed Scale it to length 1: same direction, length exactly 1.
M02.3.2-F4

The trace of the calculation: square and add, take the root, divide by the result, and check that the new vector has length 1.

Square and add: \(3^2 + 4^2 = 9 + 16 = 25\). Take the root: \(\norm{\vect{v}} = 5\). Divide: \(\hat{\vect{v}} = \tfrac{1}{5}(3, 4) = (0.6, 0.8)\). Check: \(0.6^2 + 0.8^2 = 0.36 + 0.64 = 1\).

A distance. For \(\vect{p} = (1, 1)\) and \(\vect{q} = (4, 5)\), the difference is \(\vect{q} - \vect{p} = (3, 4)\), so the distance is 5.

In space. \(\norm{(1, 2, 2)} = \sqrt{1 + 4 + 4} = 3\), and the unit vector is \(\big(\tfrac13, \tfrac23, \tfrac23\big)\). Check: \(\tfrac19 + \tfrac49 + \tfrac49 = 1\).

Scaling. \(-2\vect{v} = (-6, -8)\) has length \(\sqrt{36 + 64} = 10\), which is \(\abs{-2} \cdot 5\), as property 2 says.

Three rulers on one vector. Take \(\vect{w} = (3, -4)\).

  • 1-norm: \(\abs{3} + \abs{-4} = 7\).
  • 2-norm: \(\sqrt{9 + 16} = 5\).
  • max-norm: \(\max(3, 4) = 4\).

They come out in the promised order, \(4 \le 5 \le 7\).

Code

From scratch, with the dot product of the last lesson.

import math

def dot(a, b):
    return sum(x * y for x, y in zip(a, b))

def norm(v):
    """Euclidean length: the square root of v . v."""
    return math.sqrt(dot(v, v))

def distance(p, q):
    """The length of the arrow from p to q."""
    return norm([b - a for a, b in zip(p, q)])

def normalise(v):
    """The unit vector that points the same way as v."""
    size = norm(v)
    if size == 0:
        raise ValueError("the zero vector has no direction")
    return [x / size for x in v]

v = (3, 4)
print("length of (3, 4):", norm(v), "   in space:", norm((1, 2, 2)))
print("distance from (1, 1) to (4, 5):", distance((1, 1), (4, 5)))
print("unit vector:", normalise(v), "   its length:", norm(normalise(v)))
print("length of -2v:", norm((-6, -8)))
length of (3, 4): 5.0    in space: 3.0
distance from (1, 1) to (4, 5): 5.0
unit vector: [0.6, 0.8]    its length: 1.0
length of -2v: 10.0

The other two rulers take one line each.

def norm_1(v):
    return sum(abs(x) for x in v)

def norm_max(v):
    return max(abs(x) for x in v)

w = (3, -4)
print("1-norm:", norm_1(w), "   2-norm:", norm(w), "   max-norm:", norm_max(w))
1-norm: 7    2-norm: 5.0    max-norm: 4

The library version is np.linalg.norm. With no second argument it gives the 2-norm of a vector, and ord chooses another ruler. The loop tests the three properties and the ordering on random vectors.

import numpy as np

w = np.array([3.0, -4.0])
print("np.linalg.norm:", np.linalg.norm(w), np.linalg.norm(w, ord=1), np.linalg.norm(w, ord=np.inf))
print("unit vector:", w / np.linalg.norm(w))

rng = np.random.default_rng(0)
for _ in range(1000):
    a, b = rng.normal(size=(2, 6))                    # two vectors with six entries
    k = rng.normal()
    assert np.isclose(np.linalg.norm(k * a), abs(k) * np.linalg.norm(a))            # property 2
    assert np.linalg.norm(a + b) <= np.linalg.norm(a) + np.linalg.norm(b) + 1e-12   # property 3
    assert np.isclose(np.linalg.norm(a / np.linalg.norm(a)), 1.0)                   # normalising
    assert np.linalg.norm(a, ord=np.inf) <= np.linalg.norm(a) <= np.linalg.norm(a, ord=1)
print("scaling, triangle inequality, normalising and the ordering hold on 1000 random vectors")
np.linalg.norm: 5.0 7.0 4.0
unit vector: [ 0.6 -0.8]
scaling, triangle inequality, normalising and the ordering hold on 1000 random vectors

One place where you have already met a squared length: the sum of squared errors of a model is the squared distance between the list of targets and the list of predictions.

targets = np.array([2.0, 3.0, 5.0])
predictions = np.array([2.0, 4.0, 4.0])
print("sum of squared errors:", np.sum((targets - predictions) ** 2))
print("squared distance:     ", round(np.linalg.norm(targets - predictions) ** 2, 10))
sum of squared errors: 2.0
squared distance:      2.0

Explore

This module’s interactive, the projection explorer, lives in lesson M02.3.4. Its read-out shows the lengths of both arrows. Stretch \(\vect{b}\) without turning it and watch its length grow while the angle stays the same.

Pitfalls

Two panels, each showing a quarter of the unit circle, labelled length 1, and the direction of the vector (3, 4) as a thin grey line. Left panel, marked with a cross and titled Divide by the sum, 7: a vermillion arrow ends at the point (3 over 7, 4 over 7), which lies on a dashed straight line joining 1 on one axis to 1 on the other, well inside the circle; underneath: entries sum to 1, but length 5 over 7, about 0.71. Right panel, marked with a tick and titled Divide by the length, 5: a green arrow ends at (0.6, 0.8), exactly on the circle; underneath: 0.6 squared plus 0.8 squared equals 1: length 1.
M02.3.2-F5

Wrong beside right. Dividing \((3, 4)\) by the sum of its entries lands on the diamond of the 1-norm, short of the circle. Dividing by the length lands on the circle.

“To normalise a vector, divide it by the sum of its entries.”

That makes the entries add up to 1, which is what you want for a list of probabilities. It does not make the length 1. For \((3, 4)\) it gives \(\big(\tfrac37, \tfrac47\big)\), whose length is \(\tfrac57 \approx 0.71\). “Normalise” has both meanings in machine learning, so check which ruler a piece of code or a paper intends. A unit vector is made by dividing by the 2-norm.

“The length of \(\vect{a} + \vect{b}\) is the length of \(\vect{a}\) plus the length of \(\vect{b}\).”

Only when the two arrows point the same way. For \(\vect{a} = (3, 0)\) and \(\vect{b} = (0, 4)\) the sum has length 5, not 7. In general the triangle inequality gives \(\norm{\vect{a} + \vect{b}} \le \norm{\vect{a}} + \norm{\vect{b}}\), and for opposite arrows the sum can even be zero.

“Nearest is nearest, whatever the ruler.”

Which point is nearest can depend on the norm. Measured from the origin, \((0, 4)\) is nearer than \((3, 3)\) with the 2-norm (\(4\) against \(\sqrt{18} \approx 4.24\)), but \((3, 3)\) is nearer with the max-norm (\(3\) against \(4\)). A nearest-neighbour method inherits every such choice, so the ruler is part of the model.

Check yourself

Recall

1. What is the length of \((6, 8)\)?

Show answer

\(\sqrt{36 + 64} = \sqrt{100} = 10\).

2. How do you find the distance between two points \(\vect{p}\) and \(\vect{q}\)?

Show answer

Subtract, then measure: the distance is \(\norm{\vect{q} - \vect{p}}\).

3. What is a unit vector, and how do you make one from a non-zero vector \(\vect{v}\)?

Show answer

A vector of length 1. Divide \(\vect{v}\) by its length: \(\hat{\vect{v}} = \vect{v} / \norm{\vect{v}}\).

4. Give the 1-norm, the 2-norm and the max-norm of \((3, -4)\).

Show answer

\(7\), \(5\) and \(4\).

5. What are the unit shapes of those three norms in the plane?

Show answer

A diamond for the 1-norm, a circle for the 2-norm and a square for the max-norm.

Exercises

1. ★ Find the length of \((2, -1, 2)\) and the unit vector that points the same way.

Show solution

\(\norm{(2, -1, 2)} = \sqrt{4 + 1 + 4} = \sqrt{9} = 3\). The unit vector is \(\tfrac13(2, -1, 2) = \big(\tfrac23, -\tfrac13, \tfrac23\big)\). Check: \(\tfrac49 + \tfrac19 + \tfrac49 = 1\).

2. ★★ (a) Find the distance between \(\vect{p} = (2, -1, 3)\) and \(\vect{q} = (5, 3, 3)\). (b) Which of the points \((4, 1)\) and \((3, 3)\) is nearer to the origin under the 1-norm? Under the 2-norm? Under the max-norm?

Show solution

(a) \(\vect{q} - \vect{p} = (3, 4, 0)\), so the distance is \(\sqrt{9 + 16 + 0} = 5\).

(b) For \((4, 1)\) the three norms are \(4 + 1 = 5\), \(\sqrt{17} \approx 4.12\) and \(4\). For \((3, 3)\) they are \(6\), \(\sqrt{18} \approx 4.24\) and \(3\). So \((4, 1)\) is nearer under the 1-norm (\(5 < 6\)) and under the 2-norm (\(4.12 < 4.24\)), but \((3, 3)\) is nearer under the max-norm (\(3 < 4\)).

3. ★★★ Prove that \(\norm{\vect{v}}_\infty \le \norm{\vect{v}}_2 \le \norm{\vect{v}}_1\) for every vector \(\vect{v}\) in \(\R^n\). When are all three equal?

Show solution

First inequality. Let \(v_j\) be an entry of largest size, so \(\norm{\vect{v}}_\infty = \abs{v_j}\). The sum \(v_1^2 + \cdots + v_n^2\) contains the term \(v_j^2\), and its other terms are not negative. So \(\norm{\vect{v}}_2^2 \ge v_j^2 = \norm{\vect{v}}_\infty^2\). Both norms are non-negative, so taking square roots keeps the order: \(\norm{\vect{v}}_2 \ge \norm{\vect{v}}_\infty\).

Second inequality. Square the 1-norm and multiply out:

\(\norm{\vect{v}}_1^2 = \big(\abs{v_1} + \cdots + \abs{v_n}\big)^2 = \big(v_1^2 + \cdots + v_n^2\big) + (\text{products } \abs{v_i}\,\abs{v_k} \text{ with } i \ne k)\).

The first bracket is \(\norm{\vect{v}}_2^2\) and the products are not negative. So \(\norm{\vect{v}}_1^2 \ge \norm{\vect{v}}_2^2\), and taking square roots gives \(\norm{\vect{v}}_1 \ge \norm{\vect{v}}_2\).

Equality. If at most one entry is non-zero, all three norms equal the size of that entry. If two entries \(v_i\) and \(v_k\) are non-zero, the product \(\abs{v_i}\,\abs{v_k}\) is positive, so \(\norm{\vect{v}}_1 > \norm{\vect{v}}_2\). So all three are equal exactly when the vector has at most one non-zero entry: it lies along an axis, where the diamond, the circle and the square touch.

Summary card

Length is Pythagoras, written as a dot product. Distance is the length of a difference.

The anchor figure again: on the left the arrow (3, 4) as the long side of a right triangle with short sides 3 and 4, so its length is 5; on the right the points (1, 1) and (4, 5), whose difference is the same arrow (3, 4), so their distance is 5.
M02.3.2-F1

Square, add, take the root.

  • Length. \(\norm{\vect{v}} = \sqrt{\vect{v} \cdot \vect{v}} = \sqrt{v_1^2 + \cdots + v_n^2}\).
  • Distance. The distance from \(\vect{p}\) to \(\vect{q}\) is \(\norm{\vect{q} - \vect{p}}\).
  • Unit vector. \(\hat{\vect{v}} = \vect{v} / \norm{\vect{v}}\) for \(\vect{v} \ne \vect{0}\).
  • Properties. \(\norm{k\vect{v}} = \abs{k}\,\norm{\vect{v}}\) and \(\norm{\vect{a} + \vect{b}} \le \norm{\vect{a}} + \norm{\vect{b}}\).
  • Other rulers. \(\norm{\vect{v}}_1 = \abs{v_1} + \cdots + \abs{v_n}\) and \(\norm{\vect{v}}_\infty = \max_i \abs{v_i}\), with \(\norm{\vect{v}}_\infty \le \norm{\vect{v}}_2 \le \norm{\vect{v}}_1\).

Sources and next steps

Sources. Each was opened and checked against this lesson on 12 November 2026.

  • Marc Deisenroth, Aldo Faisal and Cheng Soon Ong, Mathematics for Machine Learning, Cambridge University Press, 2020 (free PDF). Section 3.1: Definition 3.1 lists the three properties of a norm; Examples 3.1 and 3.2 define the Manhattan norm (3.3) and the Euclidean norm (3.4), and Figure 3.3 draws their unit shapes. Section 3.3: the norm that comes from an inner product (3.16) and the distance \(\norm{\vect{x} - \vect{y}}\) (Definition 3.6).
  • Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, chapter 2, MIT Press, 2016. Section 2.5: the \(L^1\) norm (2.31) and the max norm (2.32), with where machine learning uses each.
  • Sheldon Axler, Linear Algebra Done Right, 4th edition (open access). Section 6A: the norm defined as \(\sqrt{\langle v, v\rangle}\) (6.7) and the proof that \(\norm{\lambda v} = \abs{\lambda}\,\norm{v}\) by working with the square (6.9).
  • NumPy, numpy.linalg.norm. With ord=None a vector gets its 2-norm; ord=1 and ord=np.inf give the other two rulers.
  • scikit-learn, user guide, Section 1.1, Linear Models. Ridge regression penalises \(\norm{\vect{w}}_2^2\) (Section 1.1.2) and the lasso penalises \(\norm{\vect{w}}_1\) and can set coefficients exactly to zero (Section 1.1.3).

Next. M02.3.3 Angles, orthogonality and cosine similarity divides a dot product by two lengths to measure direction alone, and proves the triangle inequality on the way.

This lesson unlocks

M02.3.3 Angles, orthogonality and cosine similarity

Your progress is saved in this browser only. No account, no tracking.
Back to top