\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M02.10 · Orthogonality, Gram-Schmidt and QR

A free, illustrated module of 4 lessons and one interactive in M02 Linear Algebra I: Vectors and Matrices: Orthonormal bases; Orthogonal matrices; Gram-Schmidt; QR and least squares.

M02.10 L1 · Core 4 lessons · about 100 min Live · in review

Part of M02 Linear Algebra I: Vectors and Matrices in the Mathematics track.

Perpendicular unit vectors are the easiest basis there is: every coordinate is a single dot product and every length can be read straight off the numbers. These four lessons show why, meet the matrices built from such bases (the rotations and reflections, whose inverse is simply their transpose), turn any independent set into an orthonormal one by subtracting shadows, and record that recipe as the factorisation A = QR, which is how a computer fits a line to data.

What you need first. Dot products, lengths and projection onto a line (M02.3), basis and linear independence (M02.2), matrices as maps with the rotation matrix (M02.4), and coordinates in a new basis (M02.9). The inverse (M02.7) and the determinant (M02.8) are used in the second lesson, and back substitution (M02.6) in the last.

Lessons

1. Orthonormal bases M02.10.1 25 min

   Perpendicular unit vectors are the easiest basis to work in: each coordinate is one dot product, with no system of equations to solve. Derive that rule, adapt it to bases that are only orthogonal, and prove that perpendicular vectors, none of them zero, are always independent.

2. Orthogonal matrices M02.10.2 25 min

   Put an orthonormal basis in the columns of a matrix and you get a map that moves space rigidly. See why such a matrix satisfies Q-transpose times Q equals I, prove that it keeps every length and angle, and show that it is a rotation or a reflection whose inverse is its transpose.

3. Gram-Schmidt M02.10.3 25 min

   Any independent set of vectors can be straightened into an orthonormal set that spans the same space. Learn the two moves of the Gram-Schmidt process, subtracting shadows and scaling to length one, work it by hand in two and three dimensions, and code it from scratch.

4. QR and least squares M02.10.4 25 min

   Gram-Schmidt, written down as it runs, factors a matrix as A = QR: orthonormal columns times a triangle. Use the factorisation to project a vector onto a subspace and to fit the least-squares line through data, and check the answer against numpy.linalg.lstsq.

The interactive

M02.10-X1 · Gram-Schmidt stepper. Drag two vectors in the plane, or three in space, and step through Gram-Schmidt: scale the first to length one, subtract each shadow, and watch what is left turn into a perpendicular unit vector. It lives in Gram-Schmidt.