\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M02.3 · Dot product, norms, angles and projections

A free, illustrated module of 4 lessons and one interactive in M02 Linear Algebra I: Vectors and Matrices: The dot product, two ways; Length, distance and norms; Angles, orthogonality and cosine similarity; Projection onto a line.

M02.3 L1 · Core 4 lessons · about 100 min Live · in review

Part of M02 Linear Algebra I: Vectors and Matrices in the Mathematics track.

One small operation, multiply matching entries and add, turns out to measure everything geometric about vectors. These four lessons show the dot product two ways and prove that they agree, use it to define length, distance and other norms, turn it into angles and cosine similarity with the Cauchy-Schwarz inequality behind them, and end with the shadow of one vector on another: the projection, which is the closest point on a line and the seed of least squares.

What you need first. Vectors as arrows and as lists of numbers, adding, subtracting and scaling them (M02.1), and the cosine of an angle from the unit circle (M01.5). Linear combinations (M02.2) are mentioned once and are not required.

Lessons

1. The dot product, two ways M02.3.1 25 min

   The dot product turns two vectors into one number. Compute it by multiplying matching coordinates and adding, or as length times length times the cosine of the angle, and see from the law of cosines why the two recipes always agree.

2. Length, distance and norms M02.3.2 25 min

   The length of a vector is Pythagoras written with a dot product, and the distance between two points is the length of their difference. Learn to normalise a vector, and meet two other rulers, the 1-norm and the max-norm, through their unit shapes.

3. Angles, orthogonality and cosine similarity M02.3.3 25 min

   Divide a dot product by two lengths and you get the cosine of the angle between two vectors: a measure of direction that ignores size. Learn the perpendicularity test, prove the Cauchy-Schwarz inequality that makes angles work in any dimension, and compare documents by cosine similarity.

4. Projection onto a line M02.3.4 25 min

   The projection of a vector onto a line is its shadow: the point of the line closest to it. Derive the formula by making the error perpendicular to the line, split any vector into a parallel and a perpendicular part, and see why this is the seed of least squares.

The interactive

M02.3-X1 · Projection explorer. Drag a vector and a direction and watch the shadow, the error and the right angle between them follow. The dot product, the angle, the cosine and the projection update as you move, and a guess slider lets you check that no other point of the line is closer. It lives in Projection onto a line.