Skip to main content

The inverse: undoing a map

A linear map can be undone exactly when it loses nothing: no two inputs share an output. Meet the inverse matrix, derive the formula for a 2-by-2 inverse step by step, prove that the inverse of a product reverses the order, and see why a map that squashes can never be run backwards.
M02.7.1L1 Core25 minIn review

Before this lesson

M02.4 Matrices as linear maps M02.5 Matrix multiplication as composition M02.6 Linear systems and elimination

After this lesson you can

  • State the two products that define the inverse of a matrix, and explain in a picture what the inverse does.
  • Decide whether a 2-by-2 matrix is invertible by computing ad - bc, and write down its inverse when it is.
  • Explain why a map that sends two different inputs to the same output cannot be undone.
  • Prove that the inverse of a product is the product of the inverses in the opposite order.

In review. The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

The question

A layer of a neural network turns an input into an output. Suppose you are handed only the output. Can you work out the input it came from? For some weight matrices you always can: a second matrix runs the first one backwards. For others the input is gone for good. What decides which kind of matrix you have, and how do you find the one that runs it backwards?

Intuition

A matrix is a map: it moves every point of the plane at once (M02.4). To undo a map is to move every point back to where it started. The map that does this is called the inverse.

Two square panels side by side. The left panel, titled The input plane, shows a blue unit grid, a blue letter F and a blue dot labelled v equals (2, 1). The right panel, titled The output plane, shows the same grid after the map, now a vermillion grid of slanted parallelograms, with a slanted vermillion letter F and a vermillion dot labelled T of v equals (3, 0). A green arrow labelled T points from the left panel to the right panel, and a second green arrow labelled T inverse points from the right panel back to the left. A sentence underneath reads: T inverse sends every point back to where T took it from.
M02.7.1-F1

The running map of this course, \(T(x, y) = {(x + y,\; -x + 2y)}\), carries the grid on the left to the grid on the right. Its inverse \(T^{-1}\) carries everything back. The point \(\vect{v} = (2, 1)\) goes to \((3, 0)\) and returns.

Do one map and then the other, and nothing has moved. “Nothing has moved” is itself a map, the identity (M02.5). So undoing has an exact meaning: \(T^{-1}\) after \(T\) is the identity.

Not every map has an inverse. The rest of this lesson finds out which do, and builds the inverse when it exists.

Build it up

Step 1. Undoing means getting every input back. Write \(\mat{A}\) for the matrix of \(T\), and \(\mat{B}\) for the matrix of a map that undoes it. “Apply \(\mat{A}\), then \(\mat{B}\)” is the product \(\mat{B}\mat{A}\) (M02.5). Getting every input back means \(\mat{B}\mat{A}\vect{v} = \vect{v}\) for every \(\vect{v}\). Only one matrix leaves every vector alone, the identity \(\mat{I}\), so

\[ \mat{B}\mat{A} = \mat{I} . \]

Step 2. A map that loses information cannot be undone. Take the matrix behind the Squash onto a line button of the matrix playground in M02.4.2:

\[ \mat{S} = \begin{bmatrix} 1 & 2 \\ 0.5 & 1 \end{bmatrix} . \]

Its columns are \((1, 0.5)\) and \((2, 1)\). The second is twice the first, so every output \({x\,(1, 0.5) + y\,(2, 1)}\) is a multiple of \((1, 0.5)\). The whole plane lands on one line.

Two panels. The left panel, titled Three different inputs, shows three blue dots at (2, 0), (0, 1) and (minus 2, 2), joined by a dashed blue line. A green arrow labelled S points to the right panel, titled One output. There a vermillion line runs through the origin, with the note every output lies on this line, and a single vermillion dot sits on it at (2, 1). A dashed grey arrow with a question mark points from the right panel back to the left. A sentence underneath reads: given the output (2, 1), no rule can say which input it came from.
M02.7.1-F2

The squash \(\mat{S}\) sends three different inputs to the same output. An inverse would have to send \((2, 1)\) back to all three at once, and no function can do that.

Check the three inputs by combining columns. For the third, \(-2\) of the first column is \((-2, -1)\) and \(2\) of the second is \((4, 2)\).

\[ \begin{aligned} \mat{S}(2, 0) &= 2\,(1, 0.5) = (2, 1), \\ \mat{S}(0, 1) &= 1\,(2, 1) = (2, 1), \\ \mat{S}(-2, 2) &= (-2, -1) + (4, 2) \\ &= (2, 1). \end{aligned} \]

Once three inputs have become one output, the information about which one you started from no longer exists. So a map can be undone only if different inputs always give different outputs. We will say that such a map loses nothing.

Step 3. The columns of the inverse answer two questions. Back to \(\mat{A}\), which loses nothing. The first column of any matrix is where it sends \(\ihat = (1, 0)\) (M02.4.2). So the first column of \(\mat{B}\) is the input that \(\mat{A}\) sends to \(\ihat\). Find it by solving \(\mat{A}\vect{x} = \ihat\):

\[ \begin{aligned} x + y &= 1 \\ -x + 2y &= 0 \end{aligned} \]

Add the two equations: \(3y = 1\), so \(y = \tfrac13\). The first equation then gives \(x = {1 - \tfrac13} = \tfrac23\).

The second column is the input sent to \(\jhat = (0, 1)\). Solve \({x + y} = 0\) and \({-x + 2y} = 1\): adding gives \(3y = 1\) again, so \(y = \tfrac13\) and \(x = -\tfrac13\). Side by side,

\[ \mat{B} = \begin{bmatrix} \tfrac23 & -\tfrac13 \\[2pt] \tfrac13 & \tfrac13 \end{bmatrix}. \]

Test it on the figure’s point: \(\mat{B}(3, 0) = 3\,(\tfrac23, \tfrac13) = (2, 1)\). It works.

Step 4. The same calculation with letters gives a formula. That is the next section.

The formal statement

NoteDefinition: the inverse of a matrix

A square matrix \(\mat{A}\) is invertible if there is a matrix \(\mat{B}\) of the same size with

\[ \mat{B}\mat{A} = \mat{I} \qquad \text{and} \qquad \mat{A}\mat{B} = \mat{I} . \]

Then \(\mat{B}\) is called the inverse of \(\mat{A}\) and is written \(\mat{A}^{-1}\). A square matrix with no inverse is called singular.

The first product says that \(\mat{B}\) undoes \(\mat{A}\). The second says that \(\mat{A}\) undoes \(\mat{B}\). For square matrices each one forces the other, which lesson M02.7.4 proves.

There is only one inverse. Suppose \(\mat{B}\) and \(\mat{C}\) are both inverses of \(\mat{A}\). Then

\[ \begin{aligned} \mat{B} &= \mat{B}\mat{I} = \mat{B}(\mat{A}\mat{C}) \\ &= (\mat{B}\mat{A})\mat{C} = \mat{I}\mat{C} = \mat{C}. \end{aligned} \]

The second step uses \(\mat{A}\mat{C} = \mat{I}\), the third moves the brackets (products are associative, M02.5.3), and the fourth uses \(\mat{B}\mat{A} = \mat{I}\). So we may say the inverse.

NoteResult: the inverse of a 2-by-2 matrix

Let \(\mat{A} = \begin{bmatrix} a & b \\ c & d \end{bmatrix}\). If \({ad - bc} \neq 0\), then

\[ \mat{A}^{-1} = \frac{1}{ad - bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}. \]

If \({ad - bc} = 0\), then \(\mat{A}\) has no inverse.

Derivation. Solve \(\mat{A}\vect{x} = (p, q)\) for any right-hand side, as in Step 3:

\[ \begin{aligned} ax + by &= p && (1) \\ cx + dy &= q && (2) \end{aligned} \]

Take \(d\) times (1) minus \(b\) times (2). The \(y\) terms are \({bdy - bdy} = 0\), which leaves \({(ad - bc)\,x} = {dp - bq}\). Take \(a\) times (2) minus \(c\) times (1). The \(x\) terms are \({acx - acx} = 0\), which leaves \({(ad - bc)\,y} = {aq - cp}\). If \({ad - bc} \neq 0\), divide:

\[ \begin{aligned} \begin{bmatrix} x \\ y \end{bmatrix} &= \frac{1}{ad - bc} \begin{bmatrix} dp - bq \\ -cp + aq \end{bmatrix} \\[4pt] &= \frac{1}{ad - bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix} \begin{bmatrix} p \\ q \end{bmatrix}. \end{aligned} \]

This shows what the inverse must be. To be sure that it is one, multiply out both products. First \(\mat{A}\) times the new matrix:

\[ \begin{aligned} &\begin{bmatrix} a & b \\ c & d \end{bmatrix} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix} \\[4pt] &\quad = \begin{bmatrix} ad - bc & -ab + ba \\ cd - dc & -cb + da \end{bmatrix} \\[4pt] &\quad = (ad - bc)\,\mat{I}. \end{aligned} \]

Then the new matrix times \(\mat{A}\):

\[ \begin{aligned} &\begin{bmatrix} d & -b \\ -c & a \end{bmatrix} \begin{bmatrix} a & b \\ c & d \end{bmatrix} \\[4pt] &\quad = \begin{bmatrix} da - bc & db - bd \\ -ca + ac & -cb + ad \end{bmatrix} \\[4pt] &\quad = (ad - bc)\,\mat{I}. \end{aligned} \]

Dividing by \({ad - bc}\) turns both right-hand sides into \(\mat{I}\), as the definition demands.

When \({ad - bc} = 0\). The first product above is then the zero matrix. Read it one column at a time: \(\mat{A}(d, -c) = \vect{0}\) and \(\mat{A}(-b, a) = \vect{0}\). If \(\mat{A}\) had an inverse, we could apply it to both equations and get \((d, -c) = \vect{0}\) and \((-b, a) = \vect{0}\), so all four entries would be zero. But the zero matrix times anything is the zero matrix, never \(\mat{I}\). So there is no inverse.

The number \({ad - bc}\) is the determinant of M02.8, where it is shown to be the factor by which the map scales areas. The matrix playground of M02.4.2 prints it as the area factor. Zero area means the plane has been squashed onto a line or a point.

Invertible means nothing is lost and nothing is missed. Let \(\mat{A}\) be invertible.

  • If \(\mat{A}\vect{u} = \mat{A}\vect{v}\), apply \(\mat{A}^{-1}\) to both sides: \(\vect{u} = \vect{v}\). Different inputs give different outputs.
  • Every \(\vect{b}\) is an output, because \(\mat{A}(\mat{A}^{-1}\vect{b}) = \vect{b}\).

Together: the system \(\mat{A}\vect{x} = \vect{b}\) has exactly one solution, \(\vect{x} = \mat{A}^{-1}\vect{b}\). For a 2-by-2 matrix the converse holds too. If \({ad - bc} = 0\), we have just seen a non-zero vector sent to \(\vect{0}\) (or else \(\mat{A}\) is the zero matrix), so something is lost. Hence a matrix that loses nothing has \({ad - bc} \neq 0\) and is invertible. M02.7.4 proves the same for square matrices of every size.

NoteResult: the inverse of a product reverses the order

If \(\mat{A}\) and \(\mat{B}\) are invertible matrices of the same size, then so is \(\mat{A}\mat{B}\), and

\[ (\mat{A}\mat{B})^{-1} = \mat{B}^{-1}\mat{A}^{-1} . \]

Proof. Multiply the claimed inverse by \(\mat{A}\mat{B}\) on each side, moving brackets by associativity:

\[ \begin{aligned} (\mat{A}\mat{B})(\mat{B}^{-1}\mat{A}^{-1}) &= \mat{A}(\mat{B}\mat{B}^{-1})\mat{A}^{-1} \\ &= \mat{A}\,\mat{I}\,\mat{A}^{-1} \\ &= \mat{A}\mat{A}^{-1} = \mat{I}, \\[6pt] (\mat{B}^{-1}\mat{A}^{-1})(\mat{A}\mat{B}) &= \mat{B}^{-1}(\mat{A}^{-1}\mat{A})\mat{B} \\ &= \mat{B}^{-1}\,\mat{I}\,\mat{B} \\ &= \mat{B}^{-1}\mat{B} = \mat{I}. \end{aligned} \]

Both products are \(\mat{I}\), so \(\mat{B}^{-1}\mat{A}^{-1}\) is the inverse of \(\mat{A}\mat{B}\). In words: \(\mat{A}\mat{B}\) means “\(\mat{B}\) first, then \(\mat{A}\)”, and to undo that you undo the last step first. You put on socks, then shoes, and you take off shoes, then socks.

Go deeper: larger matrices: solve for all the columns at once

Step 3 found the inverse by solving two systems with the same left-hand side. Elimination (M02.6) can solve both in one pass. Write \(\mat{A}\) with the identity beside it and apply row operations until the left half is the identity:

\[ \begin{aligned} &\left[\begin{array}{rr|rr} 1 & 1 & 1 & 0 \\ -1 & 2 & 0 & 1 \end{array}\right] \\[4pt] \to\;&\left[\begin{array}{rr|rr} 1 & 1 & 1 & 0 \\ 0 & 3 & 1 & 1 \end{array}\right] \\[4pt] \to\;&\left[\begin{array}{rr|rr} 1 & 1 & 1 & 0 \\ 0 & 1 & \tfrac13 & \tfrac13 \end{array}\right] \\[4pt] \to\;&\left[\begin{array}{rr|rr} 1 & 0 & \tfrac23 & -\tfrac13 \\ 0 & 1 & \tfrac13 & \tfrac13 \end{array}\right]. \end{aligned} \]

The steps are: add row 1 to row 2, divide row 2 by 3, subtract row 2 from row 1. Each column of the right half is being solved for at the same time, so the right half ends as \(\mat{A}^{-1}\). This is Gauss-Jordan elimination, and it works for a square matrix of any size. If a pivot is missing, the left half cannot become the identity and the matrix is singular.

Frontier note: networks built to run backwards

Most networks throw information away on purpose. A flow-based generative model does the opposite: every layer is an invertible map, so an image can be turned into a code and the code turned back into exactly the same image. Glow (Kingma and Dhariwal, 2018) includes a layer that multiplies the channels at every pixel by one square weight matrix \(\mat{W}\). Table 1 of the paper lists the layer as \(\vect{y} = \mat{W}\vect{x}\) and its reverse as \(\vect{x} = \mat{W}^{-1}\vect{y}\), and the training objective includes \(\log \abs{\det \mat{W}}\), which is finite only while \(\mat{W}\) stays invertible.

Worked example

Find the inverse of \(\mat{A} = \begin{bmatrix} 1 & 1 \\ -1 & 2 \end{bmatrix}\), check it, and use it.

A diagram in two parts. The top part, headed The recipe, shows four matrices in a row joined by green arrows numbered 1, 2 and 3. First the matrix A with rows 1, 1 and minus 1, 2. After arrow 1, the matrix with rows 2, 1 and minus 1, 1, its two diagonal entries highlighted, captioned swap the two diagonal entries. After arrow 2, the matrix with rows 2, minus 1 and 1, 1, its other two entries highlighted, captioned flip the signs of the other two. After arrow 3, one third times that matrix, labelled A inverse, captioned divide by a d minus b c, which equals 1 times 2 minus 1 times minus 1, equals 3. The bottom part, headed Use it: which input did T send to (3, 0)?, shows one third times the matrix with rows 2, minus 1 and 1, 1, times the column 3, 0, equals one third of the column 6, 3, equals the column 2, 1 in a yellow box, with the note the input: T of (2, 1) equals (3, 0).
M02.7.1-F3

The recipe for a 2-by-2 inverse in three moves, and the inverse at work: it recovers the input \((2, 1)\) from the output \((3, 0)\).

The recipe. Here \(a = 1\), \(b = 1\), \(c = -1\), \(d = 2\), so \({ad - bc} = {1 \cdot 2 - 1 \cdot (-1)} = 3\). It is not zero, so the inverse exists. Swap \(a\) and \(d\), flip the signs of \(b\) and \(c\), and divide by 3:

\[ \mat{A}^{-1} = \frac13 \begin{bmatrix} 2 & -1 \\ 1 & 1 \end{bmatrix}. \]

This is the matrix \(\mat{B}\) that Step 3 found by solving two systems.

Check. Multiply, row times column:

\[ \begin{aligned} \mat{A}\mat{A}^{-1} &= \frac13 \begin{bmatrix} 1 & 1 \\ -1 & 2 \end{bmatrix} \begin{bmatrix} 2 & -1 \\ 1 & 1 \end{bmatrix} \\[4pt] &= \frac13 \begin{bmatrix} 2 + 1 & -1 + 1 \\ -2 + 2 & 1 + 2 \end{bmatrix} \\[4pt] &= \frac13 \begin{bmatrix} 3 & 0 \\ 0 & 3 \end{bmatrix} = \mat{I}. \end{aligned} \]

Use it. Which input did \(T\) send to \((3, 0)\)? Multiply the output by the inverse:

\[ \begin{aligned} \mat{A}^{-1} \begin{bmatrix} 3 \\ 0 \end{bmatrix} &= \frac13 \begin{bmatrix} 2 \cdot 3 - 1 \cdot 0 \\ 1 \cdot 3 + 1 \cdot 0 \end{bmatrix} \\[4pt] &= \frac13 \begin{bmatrix} 6 \\ 3 \end{bmatrix} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}. \end{aligned} \]

The system \(\mat{A}\vect{x} = (5, 1)\) is solved the same way:

\[ \vect{x} = \mat{A}^{-1} \begin{bmatrix} 5 \\ 1 \end{bmatrix} = \frac13 \begin{bmatrix} 10 - 1 \\ 5 + 1 \end{bmatrix} = \begin{bmatrix} 3 \\ 2 \end{bmatrix}. \]

This agrees with M02.4.2, where \(\mat{A}(3, 2) = (5, 1)\).

A matrix with no inverse. For the squash \(\mat{S}\), \({ad - bc} = {1 \cdot 1 - 2 \cdot 0.5} = 0\). The derivation says that \((d, -c) = (1, -0.5)\) is then sent to zero, and it is:

\[ \begin{aligned} &\mat{S}(1, -0.5) \\ &\quad = 1\,(1, 0.5) - 0.5\,(2, 1) \\ &\quad = (1, 0.5) - (1, 0.5) = (0, 0). \end{aligned} \]

Two inputs, \((1, -0.5)\) and \((0, 0)\), share the output \((0, 0)\).

Code

From scratch first. Exact fractions keep the thirds as thirds.

from fractions import Fraction

def inverse_2x2(A):
    """The inverse of a 2 by 2 matrix by the formula. A is a list of two rows."""
    (a, b), (c, d) = A
    det = a * d - b * c
    if det == 0:
        raise ValueError("ad - bc = 0, so this matrix has no inverse")
    return [[Fraction(d) / det, Fraction(-b) / det],
            [Fraction(-c) / det, Fraction(a) / det]]

def matmul(A, B):
    """The row-times-column product of two 2 by 2 matrices."""
    return [[sum(A[i][k] * B[k][j] for k in range(2)) for j in range(2)] for i in range(2)]

def show(M):
    return [[str(x) for x in row] for row in M]

A = [[1, 1], [-1, 2]]
A_inv = inverse_2x2(A)
print("inverse of A:", show(A_inv))
print("A times it  :", show(matmul(A, A_inv)), "  it times A:", show(matmul(A_inv, A)))

S = [[1, 2], [Fraction(1, 2), 1]]
try:
    inverse_2x2(S)
except ValueError as error:
    print("S           : error,", error)
inverse of A: [['2/3', '-1/3'], ['1/3', '1/3']]
A times it  : [['1', '0'], ['0', '1']]   it times A: [['1', '0'], ['0', '1']]
S           : error, ad - bc = 0, so this matrix has no inverse

Now the library. np.linalg.inv handles a square matrix of any size, and raises an error for a singular one.

import numpy as np

A = np.array([[1.0, 1.0], [-1.0, 2.0]])
A_inv = np.linalg.inv(A)
print("3 * inv(A) =", (3 * A_inv).round(12).tolist())
print("inv(A) @ (3, 0) =", A_inv @ np.array([3.0, 0.0]))

b = np.array([5.0, 1.0])
print("solve(A, b) =", np.linalg.solve(A, b), "  inv(A) @ b =", A_inv @ b)

S = np.array([[1.0, 2.0], [0.5, 1.0]])
try:
    np.linalg.inv(S)
except np.linalg.LinAlgError as error:
    print("inv(S) raises LinAlgError:", error)
3 * inv(A) = [[2.0, -1.0], [1.0, 1.0]]
inv(A) @ (3, 0) = [2. 1.]
solve(A, b) = [3. 2.]   inv(A) @ b = [3. 2.]
inv(S) raises LinAlgError: Singular matrix

Both routes give \((3, 2)\). In real code prefer solve: it finds \(\vect{x}\) by elimination without ever forming the inverse, which is faster and loses less accuracy (M02.6.4).

Last, the order of a product, with a stretch \(\mat{D}\) and a quarter turn \(\mat{R}\).

D = np.array([[2.0, 0.0], [0.0, 1.0]])      # stretch: twice as wide
R = np.array([[0.0, -1.0], [1.0, 0.0]])     # a quarter turn anticlockwise
both = R @ D                                 # stretch first, then turn

reverse_order = np.linalg.inv(D) @ np.linalg.inv(R)
same_order = np.linalg.inv(R) @ np.linalg.inv(D)
print("inv(R @ D) equals inv(D) @ inv(R):", np.allclose(np.linalg.inv(both), reverse_order))
print("undone in reverse order:", (reverse_order @ both).tolist())
print("undone in the same order:", (same_order @ both).tolist())
inv(R @ D) equals inv(D) @ inv(R): True
undone in reverse order: [[1.0, 0.0], [0.0, 1.0]]
undone in the same order: [[2.0, 0.0], [0.0, 0.5]]

Undone in the same order, the plane is left twice as wide and half as tall. The figure in the pitfalls section draws exactly this.

Explore

This module’s interactive, the subspace viewer, is in lesson M02.7.4. It shows what a matrix reaches and what it destroys, which are the subjects of the next two lessons.

You can already test this lesson in the matrix playground of M02.4.2. Press Squash onto a line and read the area factor: it is \({ad - bc} = 0\). Then drag one column off the line. The moment the area factor leaves zero, the matrix has an inverse again.

Pitfalls

Two panels, each with a row of four boxes underneath. Left panel, marked with a cross and titled Undone in the same order: a dashed blue outline of a letter F at the start, and a filled vermillion letter F at the end that is twice as wide and half as tall, with the note the letter at the end: too wide and too short. The boxes underneath read stretch, turn, undo stretch, undo turn. Right panel, marked with a tick and titled Undone in reverse order: the filled green letter F sits exactly on the dashed outline, with the note back exactly where it started. The boxes underneath read stretch, turn, undo turn, undo stretch.
M02.7.1-F4

Wrong beside right. A letter is stretched and then turned. Undoing the stretch first leaves it the wrong shape. Undoing the turn first, the last thing done, brings it home.

“The inverse of \(\mat{A}\mat{B}\) is \(\mat{A}^{-1}\mat{B}^{-1}\).”

The order reverses: \((\mat{A}\mat{B})^{-1} = \mat{B}^{-1}\mat{A}^{-1}\). With the quarter turn \(\mat{R}\) and the stretch \(\mat{D}\) of the code above, \(\mat{D}^{-1}\mat{R}^{-1} = \begin{bmatrix} 0 & 0.5 \\ -1 & 0 \end{bmatrix}\) undoes \(\mat{R}\mat{D}\), while \(\mat{R}^{-1}\mat{D}^{-1} = \begin{bmatrix} 0 & 1 \\ -0.5 & 0 \end{bmatrix}\) does not. A deep network is a long product of layers. Running it backwards means inverting the last layer first.

“\(\mat{A}^{-1}\) means one over each entry.”

For numbers, \(x^{-1} = 1/x\). For matrices the superscript means “the map that undoes this one”. Taking one over each entry of \(\begin{bmatrix} 1 & 1 \\ -1 & 2 \end{bmatrix}\) gives \(\begin{bmatrix} 1 & 1 \\ -1 & 0.5 \end{bmatrix}\), and the first times the second is \(\begin{bmatrix} 0 & 1.5 \\ -3 & 0 \end{bmatrix}\), nowhere near \(\mat{I}\). NumPy makes the slip easy: 1 / A and A ** -1.0 both work entry by entry. The inverse is np.linalg.inv(A).

“A matrix with no zeros in it must be invertible.”

The squash \(\begin{bmatrix} 1 & 2 \\ 0.5 & 1 \end{bmatrix}\) has no zero entry and no inverse. The swap \(\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\) is half zeros and is invertible: it is its own inverse. What matters is whether the two columns lie on one line through the origin, which is what \({ad - bc} = 0\) tests.

Check yourself

Recall

1. Which two equations must \(\mat{B}\) satisfy to be the inverse of \(\mat{A}\)?

Show answer

\(\mat{B}\mat{A} = \mat{I}\) and \(\mat{A}\mat{B} = \mat{I}\).

2. Write the inverse of \(\begin{bmatrix} a & b \\ c & d \end{bmatrix}\).

Show answer

\(\dfrac{1}{ad - bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}\), provided \({ad - bc} \neq 0\): swap the diagonal, flip the other two signs, divide by \({ad - bc}\).

3. What does a 2-by-2 matrix with \({ad - bc} = 0\) do to the plane, and why does that rule out an inverse?

Show answer

It squashes the plane onto a line or a point, so different inputs share an output. An inverse would have to send one output back to several inputs.

4. What is \((\mat{A}\mat{B})^{-1}\) in terms of \(\mat{A}^{-1}\) and \(\mat{B}^{-1}\)?

Show answer

\(\mat{B}^{-1}\mat{A}^{-1}\). The last map applied is the first one undone.

5. If \(\mat{A}\) is invertible, how many solutions does \(\mat{A}\vect{x} = \vect{b}\) have?

Show answer

Exactly one, \(\vect{x} = \mat{A}^{-1}\vect{b}\), for every \(\vect{b}\).

Exercises

1. ★ Find the inverse of \(\mat{M} = \begin{bmatrix} 2 & 1 \\ 5 & 3 \end{bmatrix}\) and check it by multiplying.

Show solution

\({ad - bc} = {2 \cdot 3 - 1 \cdot 5} = 1\), so \(\mat{M}^{-1} = \begin{bmatrix} 3 & -1 \\ -5 & 2 \end{bmatrix}\). Check:

\[ \begin{aligned} &\begin{bmatrix} 2 & 1 \\ 5 & 3 \end{bmatrix} \begin{bmatrix} 3 & -1 \\ -5 & 2 \end{bmatrix} \\[4pt] &\quad = \begin{bmatrix} 6 - 5 & -2 + 2 \\ 15 - 15 & -5 + 6 \end{bmatrix} = \mat{I}. \end{aligned} \]

2. ★★ Let \(\mat{K} = \begin{bmatrix} 1 & 2 \\ 3 & k \end{bmatrix}\). (a) For which \(k\) is \(\mat{K}\) singular? (b) For that \(k\), find a non-zero vector that \(\mat{K}\) sends to zero. (c) Find \(\mat{K}^{-1}\) when \(k = 7\).

Show solution

(a) \({ad - bc} = {k - 6}\), which is zero exactly when \(k = 6\).

(b) The derivation says \((d, -c) = (6, -3)\) is sent to zero. Check by columns: the output is \({6\,(1, 3) - 3\,(2, 6)}\), which is \({(6, 18) - (6, 18)} = (0, 0)\). Any multiple, such as \((2, -1)\), works too.

(c) Now \({ad - bc} = {7 - 6} = 1\), so \(\mat{K}^{-1} = \begin{bmatrix} 7 & -2 \\ -3 & 1 \end{bmatrix}\). Check:

\[ \begin{aligned} &\begin{bmatrix} 1 & 2 \\ 3 & 7 \end{bmatrix} \begin{bmatrix} 7 & -2 \\ -3 & 1 \end{bmatrix} \\[4pt] &\quad = \begin{bmatrix} 7 - 6 & -2 + 2 \\ 21 - 21 & -6 + 7 \end{bmatrix} = \mat{I}. \end{aligned} \]

3. ★★★ Let \(\mat{A}\), \(\mat{B}\) and \(\mat{C}\) be invertible matrices of the same size. (a) Show that \(\mat{A}^{-1}\) is invertible and that its inverse is \(\mat{A}\). (b) Show that \((\mat{A}\mat{B}\mat{C})^{-1} = \mat{C}^{-1}\mat{B}^{-1}\mat{A}^{-1}\).

Show solution

(a) The definition is symmetric. \(\mat{A}^{-1}\mat{A} = \mat{I}\) and \(\mat{A}\mat{A}^{-1} = \mat{I}\) say that \(\mat{A}\) satisfies both equations required of an inverse of \(\mat{A}^{-1}\). The inverse is unique, so \((\mat{A}^{-1})^{-1} = \mat{A}\).

(b) Group the product as \((\mat{A}\mat{B})\mat{C}\). By the product rule, \(\mat{A}\mat{B}\) is invertible, and then so is \((\mat{A}\mat{B})\mat{C}\), with

\[ \begin{aligned} \big((\mat{A}\mat{B})\mat{C}\big)^{-1} &= \mat{C}^{-1}(\mat{A}\mat{B})^{-1} \\ &= \mat{C}^{-1}\mat{B}^{-1}\mat{A}^{-1}. \end{aligned} \]

The same argument, repeated, reverses a product of any length.

Summary card

The inverse undoes a map, and it exists exactly when the map loses nothing.

The anchor figure again: a blue grid with a letter F and the point (2, 1) on the left, the same grid after the map T on the right in vermillion with the point at (3, 0), a green arrow T going across and a green arrow T inverse coming back.
M02.7.1-F1

There and back again.

  • Definition. \(\mat{A}^{-1}\mat{A} = \mat{I}\) and \(\mat{A}\mat{A}^{-1} = \mat{I}\). There is at most one inverse.
  • 2 by 2. The inverse of \(\begin{bmatrix} a & b \\ c & d \end{bmatrix}\) is \(\dfrac{1}{ad - bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}\). There is none when \({ad - bc} = 0\).
  • Systems. If \(\mat{A}\) is invertible, \(\mat{A}\vect{x} = \vect{b}\) has the one solution \(\vect{x} = \mat{A}^{-1}\vect{b}\).
  • Products. \((\mat{A}\mat{B})^{-1} = \mat{B}^{-1}\mat{A}^{-1}\): undo the last step first.
  • In code. np.linalg.inv(A) for the inverse, np.linalg.solve(A, b) to solve a system.

Sources and next steps

Sources. Each was opened and checked against this lesson on 12 November 2026.

  • Sheldon Axler, Linear Algebra Done Right, 4th edition (open access). Section 3D: the definition of invertible and inverse with both products (3.59), the inverse is unique, with the one-line proof used here (3.60), and Exercises 3D.1 and 3D.2, which ask for \((T^{-1})^{-1} = T\) and \((ST)^{-1} = T^{-1}S^{-1}\).
  • Gilbert Strang, Multiplication and inverse matrices, MIT OpenCourseWare 18.06SC, lecture summary. A singular matrix sends some non-zero vector to zero; “A times column j of the inverse equals column j of the identity”; Gauss-Jordan elimination on a matrix with the identity beside it.
  • Grant Sanderson, Inverse matrices, column space, and null space, 3Blue1Brown, Linear Algebra, chapter 7. The inverse as the transformation that undoes another, and no inverse when space is squished into a smaller dimension.
  • Marc Deisenroth, Aldo Faisal and Cheng Soon Ong, Mathematics for Machine Learning, Section 2.2.2. Definition 2.3 of the inverse, the 2-by-2 inverse (2.24), which exists if and only if \({a_{11}a_{22} - a_{12}a_{21}} \neq 0\), and \((\mat{A}\mat{B})^{-1} = \mat{B}^{-1}\mat{A}^{-1}\) (2.27).
  • NumPy manual, numpy.linalg.inv and numpy.linalg.solve. inv raises LinAlgError if the matrix is not square or inversion fails; solve computes the solution of a full-rank square system.
  • Diederik Kingma and Prafulla Dhariwal, Glow: Generative Flow with Invertible 1x1 Convolutions, arXiv, 2018. Section 3.2 and Table 1: the invertible 1-by-1 convolution, its reverse \(\mat{W}^{-1}\), and its log-determinant \({h \cdot w \cdot \log \abs{\det \mat{W}}}\).

Next. M02.7.2 Column space: what a map can reach asks the question a singular matrix leaves open: if not every output can be reached, which ones can?

This lesson unlocks

M02.7.2 Column space: what a map can reach

Your progress is saved in this browser only. No account, no tracking.
Back to top