Skip to main content

Orthonormal bases

Perpendicular unit vectors are the easiest basis to work in: each coordinate is one dot product, with no system of equations to solve. Derive that rule, adapt it to bases that are only orthogonal, and prove that perpendicular vectors, none of them zero, are always independent.
M02.10.1L1 Core25 minIn review

Before this lesson

M02.2 Linear combinations, span and basis M02.3 Dot product, norms, angles and projections M02.9 Change of basis

After this lesson you can

  • Decide whether a set of vectors is orthogonal, orthonormal or neither by computing dot products.
  • Find the coordinates of a vector in an orthonormal basis with one dot product per coordinate, and derive why that works.
  • Adapt the rule to an orthogonal basis whose vectors are not of length one.
  • Prove that an orthogonal set of non-zero vectors is linearly independent.
  • Read the length of a vector from its coordinates in an orthonormal basis.

In review. The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

The question

In a new basis, finding the coordinates of a vector usually means solving a system of equations, with one unknown for every basis vector. Yet when principal component analysis hands you a new basis for a data set with a thousand features, nobody solves a thousand equations: each new coordinate is a single dot product. What is special about that basis, and why does the shortcut work?

Intuition

You have used one basis all your life without solving a system for it: \(\ihat = (1, 0)\) and \(\jhat = (0, 1)\). To find the first coordinate of \(\vect{v} = (2, 4)\) you read off the 2. Two properties of the standard basis make that possible, and they are so familiar that they are easy to miss:

  • the basis vectors are perpendicular to each other;
  • each of them has length one.

A basis with both properties is called orthonormal, and every such basis is as easy to use as the standard one. Turn the standard basis by any angle and you have another.

A plane with graph paper turned by about 37 degrees. Two short vermillion arrows from the origin, q1 pointing up and to the right and q2 pointing up and to the left, meet at a right angle. A blue arrow v equals (2, 4) casts two shadows, drawn as orange stripes along the two turned axes: one of length 4 along q1 and one of length 2 along q2. Dashed lines drop from the tip of v to the end of each shadow and meet it at a marked right angle. A pointer names the longer stripe: shadow of v on q1. Beside the plane three lines read: c1 equals v dot q1 equals 4, one dot product; c2 equals v dot q2 equals 2, one more; v equals 4 q1 plus 2 q2, no system to solve. A last line reads: the basis is orthonormal: a right angle, lengths 1.
M02.10.1-F1

The orthonormal basis \(\vect{q}_1 = (0.8, 0.6)\), \(\vect{q}_2 = (-0.6, 0.8)\) is the standard basis turned by about \(37°\). The coordinates of \(\vect{v} = (2, 4)\) in it are the lengths of the two shadows that \(\vect{v}\) casts on the basis vectors, and each shadow costs one dot product.

Lesson M02.3.4 found the shadow of a vector on a line. When the line is marked by a unit vector \(\vect{u}\), the shadow of \(\vect{v}\) has signed length \({\vect{v} \cdot \vect{u}}\). On graph paper turned to follow \(\vect{q}_1\) and \(\vect{q}_2\), the two shadows are “how far along \(\vect{q}_1\)” and “how far along \(\vect{q}_2\)”, which is what coordinates mean (M02.9.1). So

\[ c_1 = \vect{v} \cdot \vect{q}_1 = 4, \qquad c_2 = \vect{v} \cdot \vect{q}_2 = 2 . \]

No equations were solved. The rest of the lesson shows why this works, and what breaks when either of the two properties is missing.

Build it up

Step 1. Two words. A set of non-zero vectors is orthogonal if every pair in it is perpendicular: the dot product of any two different vectors of the set is zero (M02.3.3). It is orthonormal if, in addition, every vector has length one. “Ortho” is for the right angles. “Normal” is for normalised, which means scaled to length one (M02.3.2).

Three small planes side by side, each with two vermillion arrows from the origin. First, titled Independent: arrows b1 equals (1, minus 1) and b2 equals (1, 2) with an angle of 108 degrees marked between them; underneath, a coordinate costs: solve a system. Second, titled Orthogonal: arrows b1 equals (2, 1) and b2 equals (minus 1, 2) with a right angle marked and the note lengths root 5; underneath, one dot product, then divide. Third, titled Orthonormal: short arrows q1 and q2 with a right angle marked, both ending on a dotted unit circle, with the note lengths 1; underneath, one dot product.
M02.10.1-F2

Three bases of the plane. Left: the columns of the course’s running matrix, \((1, -1)\) and \((1, 2)\), are independent but not perpendicular. Middle: \((2, 1)\) and \((-1, 2)\) are orthogonal, with lengths \(\sqrt{5}\). Right: \(\vect{q}_1\) and \(\vect{q}_2\) are orthonormal.

Step 2. Dot both sides with a basis vector. We want the numbers \(c_1, c_2\) in \(\vect{v} = c_1\vect{q}_1 + c_2\vect{q}_2\). Take the dot product of both sides with \(\vect{q}_1\):

\[ \vect{v} \cdot \vect{q}_1 \;=\; c_1\,(\vect{q}_1 \cdot \vect{q}_1) + c_2\,(\vect{q}_2 \cdot \vect{q}_1) . \]

This used only the rules of M02.3.1: a dot product distributes over a sum, and a number can be moved out of it. Now the two properties do their work. Length one says \({\vect{q}_1 \cdot \vect{q}_1} = \norm{\vect{q}_1}^2 = 1\). The right angle says \({\vect{q}_2 \cdot \vect{q}_1} = 0\). So

\[ \vect{v} \cdot \vect{q}_1 \;=\; c_1 \cdot 1 + c_2 \cdot 0 \;=\; c_1 . \]

The unknown \(c_2\) has dropped out, and \(c_1\) stands alone. Dotting with \(\vect{q}_2\) instead isolates \(c_2\) in the same way.

Step 3. Without length one, divide. Suppose \(\vect{b}_1, \vect{b}_2\) are perpendicular but not of length one. The same move still drops \(c_2\), because \({\vect{b}_2 \cdot \vect{b}_1}\) is still zero, and one division finishes the job:

\[ \begin{aligned} \vect{v} \cdot \vect{b}_1 &= c_1\,(\vect{b}_1 \cdot \vect{b}_1), \\[0.3em] c_1 &= \frac{\vect{v} \cdot \vect{b}_1}{\vect{b}_1 \cdot \vect{b}_1} . \end{aligned} \]

This is the projection formula of M02.3.4.

Step 4. Without right angles, the shortcut is gone. If \({\vect{b}_2 \cdot \vect{b}_1} \neq 0\), the equation

\[ \vect{v} \cdot \vect{b}_1 = c_1\,(\vect{b}_1 \cdot \vect{b}_1) + c_2\,(\vect{b}_2 \cdot \vect{b}_1) \]

still holds both unknowns, and you are back to solving a system. Perpendicularity is exactly what untangles the coordinates from one another.

The formal statement

NoteDefinition: orthogonal and orthonormal

Vectors \(\vect{q}_1, \dots, \vect{q}_k\) in \(\R^n\) are orthonormal if

\[ \vect{q}_i \cdot \vect{q}_j = \begin{cases} 1 & \text{if } i = j, \\ 0 & \text{if } i \neq j. \end{cases} \]

If only the second line holds, and none of the vectors is zero, they are orthogonal. An orthonormal basis of \(\R^n\) is an orthonormal set that is also a basis of \(\R^n\).

Result 1: coordinates are dot products. If \(\vect{q}_1, \dots, \vect{q}_n\) is an orthonormal basis of \(\R^n\), then every vector \(\vect{v}\) in \(\R^n\) is

\[ \vect{v} \;=\; (\vect{v} \cdot \vect{q}_1)\,\vect{q}_1 + \cdots + (\vect{v} \cdot \vect{q}_n)\,\vect{q}_n . \]

Why. The \(\vect{q}\)’s are a basis, so there are numbers with \(\vect{v} = c_1\vect{q}_1 + \cdots + c_n\vect{q}_n\) (M02.2.4). Dot both sides with one basis vector, \(\vect{q}_i\):

\[ \begin{aligned} \vect{v} \cdot \vect{q}_i &= c_1\,(\vect{q}_1 \cdot \vect{q}_i) + \cdots \\ &\quad + c_i\,(\vect{q}_i \cdot \vect{q}_i) + \cdots \\ &\quad + c_n\,(\vect{q}_n \cdot \vect{q}_i) \\ &= c_i . \end{aligned} \]

Every term but one holds a dot product of two different basis vectors, which is zero. The surviving term is \({c_i \cdot 1}\). In the colours of the figure,

\[ \cparam{c_i} \;=\; \cdata{\vect{v}} \cdot \cmodel{\vect{q}_i} . \]

Result 2: an orthogonal basis needs one division. If \(\vect{b}_1, \dots, \vect{b}_n\) is a basis of \(\R^n\) whose vectors are orthogonal, the same argument leaves \({\vect{v} \cdot \vect{b}_i} = c_i\,{(\vect{b}_i \cdot \vect{b}_i)}\), so

\[ c_i \;=\; \frac{\vect{v} \cdot \vect{b}_i}{\vect{b}_i \cdot \vect{b}_i} . \]

Result 3: orthogonal vectors are independent. Let \(\vect{b}_1, \dots, \vect{b}_k\) be non-zero vectors, each perpendicular to all the others. Then they are linearly independent.

Proof. By M02.2.3 we must show that \(c_1\vect{b}_1 + \cdots + c_k\vect{b}_k = \vect{0}\) forces every weight to be zero. Dot both sides with \(\vect{b}_i\). On the left, all terms but one vanish, as before, leaving \(c_i\,{(\vect{b}_i \cdot \vect{b}_i)}\). On the right, \({\vect{0} \cdot \vect{b}_i} = 0\). So

\[ c_i\,\norm{\vect{b}_i}^2 = 0 . \]

Because \(\vect{b}_i\) is not the zero vector, \(\norm{\vect{b}_i}^2 > 0\), and therefore \(c_i = 0\). This holds for every \(i\), so all the weights are zero. \(\blacksquare\)

One consequence saves work. \(\R^n\) has dimension \(n\), so \(n\) independent vectors in it form a basis (M02.2.4). By Result 3, \(n\) orthonormal vectors in \(\R^n\) are automatically a basis. In the plane, three dot products settle everything: two lengths and one right angle.

Result 4: lengths can be read off the coordinates. Let \(\vect{u} = a_1\vect{q}_1 + a_2\vect{q}_2\) and \(\vect{v} = c_1\vect{q}_1 + c_2\vect{q}_2\) in an orthonormal basis. Expand the dot product term by term:

\[ \begin{aligned} \vect{u} \cdot \vect{v} &= a_1c_1\,(\vect{q}_1 \cdot \vect{q}_1) \\ &\quad + a_1c_2\,(\vect{q}_1 \cdot \vect{q}_2) \\ &\quad + a_2c_1\,(\vect{q}_2 \cdot \vect{q}_1) \\ &\quad + a_2c_2\,(\vect{q}_2 \cdot \vect{q}_2) \\ &= a_1c_1 + a_2c_2 . \end{aligned} \]

The two middle terms are zero and the outer two dot products are one. Setting \(\vect{u} = \vect{v}\) gives \(\norm{\vect{v}}^2 = c_1^2 + c_2^2\): Pythagoras still holds on the turned graph paper. In an orthonormal basis, coordinates behave exactly like ordinary coordinates.

Go deeper: Result 4 with n basis vectors

Let \(\vect{u} = \sum_{i=1}^{n} a_i\vect{q}_i\) and \(\vect{v} = \sum_{j=1}^{n} c_j\vect{q}_j\) in an orthonormal basis of \(\R^n\). Expanding the dot product gives one term for every pair \((i, j)\):

\[ \begin{aligned} \vect{u} \cdot \vect{v} &= \sum_{i=1}^{n}\sum_{j=1}^{n} a_i c_j\,(\vect{q}_i \cdot \vect{q}_j) \\ &= \sum_{i=1}^{n} a_i c_i . \end{aligned} \]

Of the \(n^2\) terms, only the \(n\) with \(i = j\) survive, each with \({\vect{q}_i \cdot \vect{q}_i} = 1\). With \(\vect{u} = \vect{v}\) this reads \(\norm{\vect{v}}^2 = c_1^2 + \cdots + c_n^2\). In a basis that is not orthonormal the cross terms \(a_ic_j\,{(\vect{b}_i \cdot \vect{b}_j)}\) do not vanish, and neither a length nor an angle can be read from the coordinates alone.

Worked example

Take \(\vect{v} = (2, 4)\) apart in the basis \(\vect{q}_1 = (0.8, 0.6)\), \(\vect{q}_2 = (-0.6, 0.8)\), and put it back.

Check the basis. Three dot products:

\[ \begin{aligned} \vect{q}_1 \cdot \vect{q}_1 &= 0.64 + 0.36 = 1, \\ \vect{q}_2 \cdot \vect{q}_2 &= 0.36 + 0.64 = 1, \\ \vect{q}_1 \cdot \vect{q}_2 &= -0.48 + 0.48 = 0 . \end{aligned} \]

The set is orthonormal, and two orthonormal vectors in the plane are a basis.

A diagram with two lanes of boxes, one for each basis vector. Left lane: a vermillion box q1 equals (0.8, 0.6); a blue arrow labelled dot with v leads down to an orange box c1 equals 1.6 plus 2.4 equals 4; a vermillion arrow labelled scale q1 by it leads down to an orange box 4 q1 equals (3.2, 2.4). Right lane: q2 equals (minus 0.6, 0.8); then c2 equals minus 1.2 plus 3.2 equals 2; then 2 q2 equals (minus 1.2, 1.6). Arrows from the two bottom boxes meet, labelled add, at a yellow box: (3.2 minus 1.2, 2.4 plus 1.6) equals (2, 4).
M02.10.1-F3

One lane for each basis vector. A dot product with \(\vect{v}\) gives the coordinate, the coordinate scales the basis vector, and the two pieces add back to \(\vect{v}\).

Take it apart. By Result 1,

\[ \begin{aligned} c_1 &= \vect{v} \cdot \vect{q}_1 = 0.8 \cdot 2 + 0.6 \cdot 4 \\ &= 1.6 + 2.4 = 4, \\[0.3em] c_2 &= \vect{v} \cdot \vect{q}_2 = -0.6 \cdot 2 + 0.8 \cdot 4 \\ &= -1.2 + 3.2 = 2 . \end{aligned} \]

Put it back. The two pieces are \(4\,\vect{q}_1 = (3.2, 2.4)\) and \(2\,\vect{q}_2 = (-1.2, 1.6)\). Their sum is \((2, 4)\), which is \(\vect{v}\).

Check the length. From the standard coordinates, \(\norm{\vect{v}}^2 = 2^2 + 4^2 = 20\). From the new ones, \(c_1^2 + c_2^2 = 16 + 4 = 20\). They agree, as Result 4 promised. (That the digits of \((2, 4)\) come back as \((4, 2)\) is a coincidence of this example. Exercise 1 has no such pattern.)

For comparison, the method of M02.9.1 would solve the system \(0.8c_1 - 0.6c_2 = 2\), \(0.6c_1 + 0.8c_2 = 4\). It has the same answer. The two dot products reached it without any elimination.

The same vector in an orthogonal basis. Take \(\vect{b}_1 = (2, 1)\) and \(\vect{b}_2 = (-1, 2)\). They are perpendicular, since \({\vect{b}_1 \cdot \vect{b}_2} = -2 + 2 = 0\), but \({\vect{b}_1 \cdot \vect{b}_1} = {\vect{b}_2 \cdot \vect{b}_2} = 5\). Result 2 divides by that 5:

\[ \begin{aligned} c_1 &= \frac{2 \cdot 2 + 1 \cdot 4}{5} = \frac{8}{5} = 1.6, \\[0.4em] c_2 &= \frac{-1 \cdot 2 + 2 \cdot 4}{5} = \frac{6}{5} = 1.2 . \end{aligned} \]

Check: \(1.6\,\vect{b}_1 = (3.2, 1.6)\) and \(1.2\,\vect{b}_2 = (-1.2, 2.4)\), and their sum is \((2, 4)\). Without the division the “coordinates” would be 8 and 6, five times too large.

Code

From scratch first, with vectors as plain tuples.

def dot(u, v):
    return sum(a * b for a, b in zip(u, v))

def is_orthonormal(vectors, tol=1e-12):
    """Every vector has length one and every pair is perpendicular."""
    for i, p in enumerate(vectors):
        for j, q in enumerate(vectors):
            target = 1.0 if i == j else 0.0
            if abs(dot(p, q) - target) > tol:
                return False
    return True

def coordinates(v, basis):
    """Coordinates of v in an orthogonal basis: one dot product each, then one division."""
    return [dot(v, b) / dot(b, b) for b in basis]

def rebuild(coords, basis):
    return [sum(c * b[k] for c, b in zip(coords, basis)) for k in range(len(basis[0]))]

q1, q2 = (0.8, 0.6), (-0.6, 0.8)
v = (2, 4)
c = coordinates(v, [q1, q2])
print("orthonormal:", is_orthonormal([q1, q2]))
print("coordinates:", [round(x, 10) for x in c])
print("rebuilt:    ", [round(x, 10) for x in rebuild(c, [q1, q2])])
print("in the basis (2, 1), (-1, 2):", [round(x, 10) for x in coordinates(v, [(2, 1), (-1, 2)])])
orthonormal: True
coordinates: [4.0, 2.0]
rebuilt:     [2.0, 4.0]
in the basis (2, 1), (-1, 2): [1.6, 1.2]

For an orthonormal basis dot(b, b) is 1, so the division changes nothing. With NumPy, put the basis vectors in the columns of a matrix \(\mat{Q}\). The rows of its transpose \(\mat{Q}\T\) are then the basis vectors, so the single product \(\mat{Q}\T\vect{v}\) computes every coordinate at once.

import numpy as np

Q = np.array([[0.8, -0.6],
              [0.6,  0.8]])            # columns: q1 and q2
v = np.array([2.0, 4.0])

print("dot products, Q.T @ v:   ", Q.T @ v)
print("solving the system Qc = v:", np.linalg.solve(Q, v))
dot products, Q.T @ v:    [4. 2.]
solving the system Qc = v: [4. 2.]

Now the claim of the opening question. Principal component analysis, which M03.6 derives, returns a new basis for a data set. Ask scikit-learn for one and test it.

from sklearn.decomposition import PCA

rng = np.random.default_rng(0)
X = rng.normal(size=(200, 5)) @ rng.normal(size=(5, 5))   # 200 data points, 5 correlated features
pca = PCA(n_components=3).fit(X)
W = pca.components_                                       # one new basis vector in each row

print(np.round(W @ W.T, 10) + 0.0)                        # every dot product between them
centred = X - X.mean(axis=0)
print("transform is dot products:", np.allclose(pca.transform(X), centred @ W.T))
[[1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]]
transform is dot products: True

The table of dot products is the identity, so the three rows of components_ are orthonormal. And transform is nothing more than the dot product of each centred data point with each of them. That is Result 1 at work: no system is solved, however many features there are. (Three vectors cannot be a basis of \(\R^5\). What their dot products give is the coordinates of each point’s shadow on a three-dimensional subspace, and M02.10.4 returns to that.)

Explore

This module’s interactive, the Gram-Schmidt stepper, lives in lesson M02.10.3. It builds an orthonormal basis out of any vectors you give it, one shadow at a time, using exactly the dot products of this lesson.

Pitfalls

Two panels, each with a blue arrow v equals (2, 4). Left panel, marked with a cross and titled Basis not perpendicular: unit vectors u1 along the horizontal axis and u2 leaning up and to the right. An orange path of 2 u1 followed by 4.4 u2 ends at a hollow dot well to the right of the tip of v, joined to it by a dashed line labelled misses v. Right panel, marked with a tick and titled Basis orthonormal: perpendicular unit vectors q1 and q2. An orange path of 4 q1 followed by 2 q2, with a right angle at the corner, ends exactly at the tip of v, labelled lands on v.
M02.10.1-F4

Wrong beside right. Both panels use the dot products of \(\vect{v}\) with two unit vectors as if they were coordinates. On the left the vectors are not perpendicular and the rebuilt point misses \(\vect{v}\). On the right they are, and it lands exactly.

“A coordinate is just the dot product with the basis vector.”

Only in an orthonormal basis. Take the unit vectors \(\vect{u}_1 = (1, 0)\) and \(\vect{u}_2 = (0.6, 0.8)\), which are not perpendicular. The dot products of \(\vect{v} = (2, 4)\) with them are \(2\) and \(4.4\), but \(2\,\vect{u}_1 + 4.4\,\vect{u}_2 = (4.64, 3.52)\), not \((2, 4)\). The true coordinates are \(-1\) and \(5\), found by solving a system. The two shadows overlap, so adding them counts part of \(\vect{v}\) twice.

“Orthogonal and orthonormal are the same thing.”

Orthogonal vectors need not have length one, and then the rule needs its division by \({\vect{b}_i \cdot \vect{b}_i}\). Machine-learning code trips on this when it takes dot products with directions that were never normalised. One naming trap makes it worse: an “orthogonal matrix” is a matrix whose columns are orthonormal. The next lesson explains.

“Perpendicular vectors are always independent.”

Only if none of them is the zero vector. The zero vector is perpendicular to everything, because \({\vect{0} \cdot \vect{v}} = 0\), yet any set that contains it is dependent. The proof of Result 3 used \(\norm{\vect{b}_i}^2 > 0\) at exactly one point, and that is where a zero vector would break it.

Check yourself

Recall

1. What two conditions make a set of vectors orthonormal?

Show answer

Every pair of different vectors is perpendicular (dot product zero), and every vector has length one (dot product with itself equal to one).

2. In an orthonormal basis \(\vect{q}_1, \dots, \vect{q}_n\), what is the coordinate of \(\vect{v}\) along \(\vect{q}_i\)?

Show answer

The dot product \({\vect{v} \cdot \vect{q}_i}\). So \(\vect{v} = {(\vect{v} \cdot \vect{q}_1)}\vect{q}_1 + \cdots + {(\vect{v} \cdot \vect{q}_n)}\vect{q}_n\).

3. How does the rule change when the basis is orthogonal but its vectors are not of length one?

Show answer

Divide by the basis vector’s dot product with itself: \(c_i = {(\vect{v} \cdot \vect{b}_i)} / {(\vect{b}_i \cdot \vect{b}_i)}\).

4. Why is a set of non-zero orthogonal vectors linearly independent?

Show answer

Dot the equation \(c_1\vect{b}_1 + \cdots + c_k\vect{b}_k = \vect{0}\) with \(\vect{b}_i\). Every term but one vanishes, leaving \(c_i\norm{\vect{b}_i}^2 = 0\), so \(c_i = 0\) for every \(i\).

5. A vector has coordinates \((4, 2)\) in an orthonormal basis. What is its length?

Show answer

\(\sqrt{4^2 + 2^2} = \sqrt{20} \approx 4.47\). In an orthonormal basis the squared length is the sum of the squared coordinates.

Exercises

1. ★ Check that \(\vect{p}_1 = (0.6, 0.8)\) and \(\vect{p}_2 = (-0.8, 0.6)\) are orthonormal. Then find the coordinates of \(\vect{w} = (3, 1)\) in that basis, and confirm them by rebuilding \(\vect{w}\).

Show solution

\({\vect{p}_1 \cdot \vect{p}_1} = 0.36 + 0.64 = 1\), \({\vect{p}_2 \cdot \vect{p}_2} = 0.64 + 0.36 = 1\) and \({\vect{p}_1 \cdot \vect{p}_2} = -0.48 + 0.48 = 0\). The coordinates are \(c_1 = {\vect{w} \cdot \vect{p}_1} = 1.8 + 0.8 = 2.6\) and \(c_2 = {\vect{w} \cdot \vect{p}_2} = -2.4 + 0.6 = -1.8\). Rebuilding: the two pieces are \(2.6\,\vect{p}_1 = (1.56, 2.08)\) and \(-1.8\,\vect{p}_2 = (1.44, -1.08)\), and their sum is \((3, 1)\).

2. ★★ Show that \(\vect{b}_1 = (1, 1, 1)\), \(\vect{b}_2 = (1, -1, 0)\) and \(\vect{b}_3 = (1, 1, -2)\) form an orthogonal basis of \(\R^3\). Find the coordinates of \(\vect{w} = (4, 0, -1)\) in it. Then scale the three vectors to make the basis orthonormal.

Show solution

The three dot products between different vectors are all zero:

\[ \begin{aligned} \vect{b}_1 \cdot \vect{b}_2 &= 1 - 1 + 0 = 0, \\ \vect{b}_1 \cdot \vect{b}_3 &= 1 + 1 - 2 = 0, \\ \vect{b}_2 \cdot \vect{b}_3 &= 1 - 1 + 0 = 0 . \end{aligned} \]

So the set is orthogonal. None of the vectors is zero, so by Result 3 they are independent, and three independent vectors in \(\R^3\) form a basis.

Their dot products with themselves are \(3\), \(2\) and \(6\). By Result 2,

\[ \begin{aligned} c_1 &= \frac{4 + 0 - 1}{3} = 1, \\[0.3em] c_2 &= \frac{4 - 0 + 0}{2} = 2, \\[0.3em] c_3 &= \frac{4 + 0 + 2}{6} = 1 . \end{aligned} \]

Check \(\vect{b}_1 + 2\,\vect{b}_2 + \vect{b}_3\) entry by entry: \(1 + 2 + 1 = 4\), then \(1 - 2 + 1 = 0\), then \(1 + 0 - 2 = -1\). That is \(\vect{w}\). Dividing each vector by its length gives the orthonormal basis \(\tfrac{1}{\sqrt{3}}(1, 1, 1)\), \(\tfrac{1}{\sqrt{2}}(1, -1, 0)\), \(\tfrac{1}{\sqrt{6}}(1, 1, -2)\).

3. ★★★ Let \(\vect{q}\) be a unit vector and \(\vect{v}\) any vector in \(\R^n\). Put \(\vect{e} = \vect{v} - {(\vect{v} \cdot \vect{q})}\,\vect{q}\). Show that \(\vect{e}\) is perpendicular to \(\vect{q}\), and deduce that \({(\vect{v} \cdot \vect{q})}^2 \le \norm{\vect{v}}^2\).

Show solution

Write \(c = {\vect{v} \cdot \vect{q}}\). Then \({\vect{e} \cdot \vect{q}} = {\vect{v} \cdot \vect{q}} - c\,{(\vect{q} \cdot \vect{q})} = c - c = 0\), so \(\vect{e}\) is perpendicular to \(\vect{q}\).

Now \(\vect{v} = c\,\vect{q} + \vect{e}\). Expand \({\vect{v} \cdot \vect{v}}\):

\[ \begin{aligned} \norm{\vect{v}}^2 &= c^2\,(\vect{q} \cdot \vect{q}) + 2c\,(\vect{q} \cdot \vect{e}) \\ &\quad + \vect{e} \cdot \vect{e} \\ &= c^2 + \norm{\vect{e}}^2 . \end{aligned} \]

Since \(\norm{\vect{e}}^2 \ge 0\), we get \(c^2 \le \norm{\vect{v}}^2\). A shadow is never longer than the vector that casts it. Subtracting a shadow to leave something perpendicular is the one move of Gram-Schmidt, in lesson M02.10.3.

Summary card

In an orthonormal basis, every coordinate is one dot product.

The anchor figure again: a blue arrow v equals (2, 4) on graph paper turned to follow two perpendicular unit vectors q1 and q2, with its two shadows of lengths 4 and 2 drawn along them and the equations c1 equals v dot q1 and c2 equals v dot q2.
M02.10.1-F1

Coordinates are shadows on perpendicular unit vectors.

  • Orthonormal. \({\vect{q}_i \cdot \vect{q}_j}\) is \(1\) when \(i = j\) and \(0\) otherwise.
  • Coordinates. \(\vect{v} = {(\vect{v} \cdot \vect{q}_1)}\vect{q}_1 + \cdots + {(\vect{v} \cdot \vect{q}_n)}\vect{q}_n\).
  • Orthogonal only. Divide: \(c_i = {(\vect{v} \cdot \vect{b}_i)} / {(\vect{b}_i \cdot \vect{b}_i)}\).
  • Independence. Non-zero orthogonal vectors are always linearly independent.
  • Lengths. \(\norm{\vect{v}}^2 = c_1^2 + \cdots + c_n^2\) in an orthonormal basis.

Sources and next steps

Sources. Each was opened and checked against this lesson on 12 November 2026.

  • Sheldon Axler, Linear Algebra Done Right, 4th edition (open access), Section 6B. Definition 6.22 (orthonormal), result 6.25 (orthonormal lists are linearly independent) and result 6.30, which states the coordinate rule \(v = \langle v, e_1\rangle e_1 + \cdots + \langle v, e_n\rangle e_n\) and the length rule (Parseval’s identity).
  • Marc Deisenroth, Aldo Faisal and Cheng Soon Ong, Mathematics for Machine Learning, Sections 3.4 and 3.5. Definition 3.7 (orthogonal and orthonormal vectors) and Definition 3.9 (orthonormal basis and orthogonal basis).
  • David Austin, Understanding Linear Algebra, Section 6.3. Proposition 6.3.5 gives the weights \({(\vect{b} \cdot \vect{w}_i)}/{(\vect{w}_i \cdot \vect{w}_i)}\) in an orthogonal basis, and Proposition 6.3.6 the independence of an orthogonal set.
  • scikit-learn, sklearn.decomposition.PCA. The attribute components_ holds the principal axes, and transform projects the data on them. That the axes are orthonormal is checked by the code cell above, not quoted from the page.

Next. M02.10.2 Orthogonal matrices puts an orthonormal basis into the columns of a matrix and asks what that matrix does to space.

This lesson unlocks

M02.10.2 Orthogonal matrices

Your progress is saved in this browser only. No account, no tracking.
Back to top