A positive definite matrix is one whose quadratic form is a bowl. Learn the definitions, prove the quick tests by eigenvalues, by a > 0 and ac - b^2 > 0, and by pivots, and see why X-transpose X is never negative and why ridge regression can always be solved.
M03.4.2L2 Intermediate25 minIn review (what this means)
The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.
Define positive definite, positive semidefinite, negative definite and indefinite through the sign of x-transpose S x.
Prove that a symmetric 2 by 2 matrix is positive definite exactly when a > 0 and ac - b^2 > 0, by completing the square.
Test a matrix of any size by its eigenvalues, its pivots or its leading principal minors, and explain why the tests agree.
Show that X-transpose X is positive semidefinite, and definite exactly when the columns of X are independent, and use it to explain why ridge regression always has a solution.
The question
Ridge regression solves \({(\mat{X}\T\mat{X} + \alpha\mat{I})\,\vect{w}} = {\mat{X}\T\vect{y}}\), and it never fails, even when two features are copies of each other and plain least squares has no unique answer. A covariance matrix never reports a negative variance, in any direction. Both facts are the same fact: the matrices involved are positive definite or semidefinite. How do you check that property, and how do you prove it?
From the last lesson
1. What does \({\vect{x}\T\mat{S}\vect{x}}\) become for \(\mat{S} = \begin{bmatrix} a & b \\ b & c \end{bmatrix}\)?
Show answer
\({ax^2 + 2bxy + cy^2}\).
2. Which signs of the eigenvalues give a bowl, a saddle and a trough?
Show answer
All positive: a bowl. Mixed signs: a saddle. All at least zero with a zero among them: a trough.
Intuition
The last lesson found three shapes. This lesson gives them names, and the names are about one question: is the form positive in every direction?
M03.4.2-F1
Three kinds of symmetric matrix, seen from above. A positive definite form rises every way, a semidefinite one has a flat direction, and an indefinite one rises one way and falls another.
A positive definite matrix gives a bowl: from the origin, every step in every direction goes up. A positive semidefinite one never goes down, but along some direction it stays flat. An indefinite one goes up somewhere and down somewhere else. The task is to tell which, quickly.
Build it up
Step 1. The eigenvalue test. By the principal axes theorem the form is the sum of \(\lambda_i y_i^2\), and \({\vect{y} = \mat{Q}\T\vect{x}}\) is zero only when \(\vect{x}\) is. So the form is positive for every \({\vect{x} \neq \vect{0}}\) exactly when every \({\lambda_i > 0}\). This test is complete, but it needs the eigenvalues, which are the hardest numbers to compute.
Step 2. Complete the square. For a 2 by 2 matrix with \({a \neq 0}\), gather the terms with \(x\) into one square. Expand \({a(x + \tfrac{b}{a}y)^2}\):
Two weighted squares. If both weights, \(a\) and \({(ac - b^2)/a}\), are positive, the form is a sum of two things that are never negative.
Step 3. The weights are the pivots. Eliminate on \(\mat{S}\): subtract \(b/a\) times row 1 from row 2. The second row becomes \({(0,\; c - b^2/a)}\). The two pivots are \(a\) and \({(ac - b^2)/a}\): the same two weights, and the multiplier \(b/a\) is the number inside the first square.
M03.4.2-F2
Elimination and completing the square are the same computation. The pivots (orange) are the weights of the squares and the multiplier (green) sits inside the first square.
Since \({a > 0}\), the second pivot is positive exactly when \({ac - b^2 > 0}\), and \({ac - b^2}\) is the determinant. So: \({a > 0}\) and \({\det \mat{S} > 0}\).
The formal statement
NoteDefinitions
A symmetric matrix \(\mat{S}\) is
positive definite if \({\vect{x}\T\mat{S}\vect{x} > 0}\) for every \({\vect{x} \neq \vect{0}}\);
positive semidefinite if \({\vect{x}\T\mat{S}\vect{x} \geq 0}\) for every \(\vect{x}\);
negative definite or semidefinite if \({-\mat{S}}\) is positive definite or semidefinite;
indefinite if the form takes both signs.
NoteTheorem: tests for positive definiteness
For a symmetric \({n \times n}\) matrix \(\mat{S}\) these statements are equivalent.
\(\mat{S}\) is positive definite.
Every eigenvalue of \(\mat{S}\) is positive.
Every pivot of elimination without row exchanges is positive.
Every leading principal minor is positive: \({D_k > 0}\) for \({k = 1, \dots, n}\), where \(D_k\) is the determinant of the top-left \({k \times k}\) block (Sylvester’s criterion).
For \({n = 2}\), tests 3 and 4 both read: \({a > 0}\) and \({ac - b^2 > 0}\).
Proof of 1 if and only if 2. Step 1 above, from the principal axes theorem. The same argument shows that \(\mat{S}\) is semidefinite exactly when every \(\lambda_i \geq 0\).
Proof of the 2 by 2 test. Suppose \({a > 0}\) and \({ac - b^2 > 0}\). In the identity of Step 2 both weights are positive, so the form is at least zero. It is zero only if \({y = 0}\) and \({x + \tfrac{b}{a}y = 0}\), which forces \({x = 0}\). So \(\mat{S}\) is positive definite. Conversely, suppose \(\mat{S}\) is positive definite. The point \({(1, 0)}\) gives the value \(a\), so \({a > 0}\). The point \({(-\tfrac{b}{a}, 1)}\) makes the first square vanish and gives \({(ac - b^2)/a}\), so this is positive too, and multiplying by \({a > 0}\) gives \({ac - b^2 > 0}\). \(\square\)
Why pivots and minors agree. Elimination without exchanges does not change the determinant of any top-left block, and the determinant of a triangular block is the product of its diagonal. So \({D_k = d_1 d_2 \cdots d_k}\) for the pivots \(d_k\), and each pivot is \({d_k = D_k / D_{k-1}}\). All pivots are positive exactly when all leading minors are. Positive pivots give a positive form by Step 2 repeated in every size: the form becomes a sum of \(d_k\) times squares. In the other direction, if \(\mat{S}\) is positive definite then so is each top-left block (feed it vectors whose later entries are zero), so each block has positive eigenvalues and \(D_k\), their product, is positive.
M03.4.2-F3
All symmetric matrices with off-diagonal entry 1. The test \({a > 0}\), \({ac - 1 > 0}\) cuts out the positive definite region. On the curve the determinant is zero: the matrix is positive semidefinite on the right branch and negative semidefinite on the left one.
Two consequences that machine learning relies on.
Gram matrices are semidefinite. For any matrix \(\mat{X}\) and any \(\vect{w}\),
It is zero exactly when \({\mat{X}\vect{w} = \vect{0}}\). So \({\mat{X}\T\mat{X}}\) is positive definite exactly when the null space of \(\mat{X}\) holds only \(\vect{0}\), that is, when the columns of \(\mat{X}\) are independent (M02.7.3). A sample covariance matrix \({\mat{C} = \mat{X}_c\T\mat{X}_c / (n - 1)}\) is a Gram matrix divided by a positive number, so it is semidefinite: the variance along any direction is never negative.
Ridge is always solvable. For \({\alpha > 0}\) and \({\vect{w} \neq \vect{0}}\),
so \({\mat{X}\T\mat{X} + \alpha\mat{I}}\) is positive definite. A positive definite matrix is invertible: if \({\mat{S}\vect{w} = \vect{0}}\) then \({\vect{w}\T\mat{S}\vect{w} = 0}\), which forces \({\vect{w} = \vect{0}}\). That is why ridge (ML01.9.1) always has exactly one solution.
Worked example
The running bowl. For \(\mat{S}\) with rows \({(2, 1)}\) and \({(1, 2)}\): the eigenvalues are 3 and 1; the leading minors are \({D_1 = 2}\) and \({D_2 = 4 - 1 = 3}\); the pivots are 2 and \({3/2}\). All positive: definite. The square completed is \({2(x + \tfrac12 y)^2 + \tfrac32 y^2}\). It is also a Gram matrix. With the running rectangular matrix \(\mat{B} = \begin{bmatrix} 1 & 1 \\ 1 & 0 \\ 0 & 1 \end{bmatrix}\), \({\mat{B}\T\mat{B} = \mat{S}}\), and indeed
positive whenever \({\vect{x} \neq \vect{0}}\), because the columns of \(\mat{B}\) are independent.
The running saddle. Eigenvalues 3 and \({-1}\); \({D_1 = 1}\) and \({D_2 = 1 - 4 = -3}\); pivots 1 and \({-3}\). The square completed is \({(x + 2y)^2 - 3y^2}\), negative at \({(-2, 1)}\), where it equals \({-3}\).
A 3 by 3 matrix.\(\mat{K} = \begin{bmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{bmatrix}\). Elimination adds \(\tfrac12\) of row 1 to row 2, giving the row \({(0, \tfrac32, -1)}\), then adds \(\tfrac23\) of the new row 2 to row 3, giving \({{(0, 0, 2 - \tfrac23)} = {(0, 0, \tfrac43)}}\). The pivots are 2, \(\tfrac32\), \(\tfrac43\), and their running products are the leading minors 2, 3, 4. The eigenvalues are \({2 - \sqrt2}\), 2 and \({2 + \sqrt2}\). Every test says definite.
M03.4.2-F4
The worked example as a trace: three matrices, three tests. The tests always agree.
Code
The three tests from scratch, then the library call. Pivots come from elimination without row exchanges; a zero or negative pivot ends the test.
import numpy as npdef pivots(S):"""Pivots of elimination without row exchanges (stops early at a pivot <= 0).""" U = np.array(S, dtype=float) n =len(U) out = []for k inrange(n): out.append(U[k, k])if U[k, k] <=0:breakfor i inrange(k +1, n): U[i] -= (U[i, k] / U[k, k]) * U[k]return outdef minors(S):return [np.linalg.det(np.asarray(S, dtype=float)[:k, :k]) for k inrange(1, len(S) +1)]def definite(S): p = pivots(S)returnlen(p) ==len(S) andmin(p) >0matrices = {"S": [[2, 1], [1, 2]],"saddle": [[1, 2], [2, 1]],"trough": [[1, 2], [2, 4]],"dome": [[-2, 1], [1, -2]],"K": [[2, -1, 0], [-1, 2, -1], [0, -1, 2]],}for name, M in matrices.items():print(f"{name:>7}: pivots {np.round(pivots(M), 4)}, minors {np.round(minors(M), 4)}, "f"eigenvalues {np.round(np.linalg.eigvalsh(M), 4)}, definite: {definite(M)}")
numpy.linalg.eigvalsh returns the eigenvalues of a symmetric matrix in ascending order, so the first one alone settles the question. Now Gram matrices, with a design matrix whose second column is twice its first.
B = np.array([[1.0, 1.0], [1.0, 0.0], [0.0, 1.0]])X = np.array([[1.0, 2.0], [2.0, 4.0], [3.0, 6.0]]) # dependent columnsG = X.T @ Xprint("B^T B =\n", B.T @ B)print("X^T X =\n", G, "\neigenvalues:", np.linalg.eigvalsh(G).round(10))print("X^T X + I eigenvalues:", np.linalg.eigvalsh(G + np.eye(2)).round(10))print("X (2, -1) =", X @ np.array([2.0, -1.0]))rng = np.random.default_rng(1)values = [w @ G @ w for w in rng.normal(size=(10000, 2))]print("smallest of 10000 random values of w^T X^T X w:", min(values))
B^T B =
[[2. 1.]
[1. 2.]]
X^T X =
[[14. 28.]
[28. 56.]]
eigenvalues: [-0. 70.]
X^T X + I eigenvalues: [ 1. 71.]
X (2, -1) = [0. 0. 0.]
smallest of 10000 random values of w^T X^T X w: 6.537567443110764e-07
The Gram matrix of the dependent columns has eigenvalues 70 and 0: semidefinite, with the flat direction \({(2, -1)}\) that \(\mat{X}\) sends to zero. Adding \(\mat{I}\) lifts both eigenvalues by 1, to 71 and 1, and the matrix becomes definite.
Explore
M03.4-X1
Bowl or saddle
Set \(a\), \(b\) and \(c\) with the sliders, or pick a preset, and watch the contour lines change from ellipses to parallel lines to hyperbolas as an eigenvalue crosses zero. The read-out puts the eigenvalues beside \(a\) and \({ac - b^2}\). Tap the map, or use the start sliders and Step, to run gradient steps downhill: they settle in a bowl and escape along a saddle.
Pitfalls
M03.4.2-F5
Wrong beside right. The signs of the entries say nothing on their own: all positive entries can make a saddle, and negative entries can sit in a bowl.
“Every entry is positive, so the matrix is positive definite.”
Positive definite is about the form, not the entries. The running saddle has entries 1, 2, 2, 1 and determinant \({-3}\). The matrix with rows \({(2, -1)}\) and \({(-1, 2)}\) has negative entries, yet its eigenvalues are 3 and 1. What matters is that the diagonal is large compared with the off-diagonal entries: \({ac > b^2}\).
“The determinant is positive, so the matrix is positive definite.”
The determinant is the product of the eigenvalues, and two negatives multiply to a positive. The dome with rows \({(-2, 1)}\) and \({(1, -2)}\) has determinant 3 and eigenvalues \({-1}\) and \({-3}\). That is why the test needs \({a > 0}\) as well: every leading minor, not only the last.
“For semidefinite, check that the leading minors are at least zero.”
That is not enough. The matrix with rows \({(0, 0)}\) and \({(0, -1)}\) has leading minors 0 and 0, yet \({\vect{x}\T\mat{S}\vect{x} = -y^2}\) is negative. Semidefiniteness needs every principal minor (every block on the diagonal, not only the top-left ones) to be at least zero, or simply every eigenvalue. In floating point, test against a small tolerance: eigvalsh may return a value such as \({-10^{-15}}\) for a matrix that is exactly semidefinite.
Check yourself
Recall
1. Define a positive definite matrix.
Show answer
A symmetric matrix \(\mat{S}\) with \({\vect{x}\T\mat{S}\vect{x} > 0}\) for every \({\vect{x} \neq \vect{0}}\).
2. What is the quick test for a symmetric 2 by 2 matrix?
Show answer
\({a > 0}\) and \({ac - b^2 > 0}\): a positive top-left entry and a positive determinant.
3. How are the pivots related to the leading principal minors?
Show answer
Each minor is the product of the pivots so far, \({D_k = d_1 \cdots d_k}\), so \({d_k = D_k/D_{k-1}}\).
4. Why is \({\mat{X}\T\mat{X}}\) always positive semidefinite?
Show answer
Because \({\vect{w}\T\mat{X}\T\mat{X}\vect{w} = \norm{\mat{X}\vect{w}}^2 \geq 0}\).
5. When is \({\mat{X}\T\mat{X}}\) positive definite?
Show answer
Exactly when the columns of \(\mat{X}\) are independent, so that \({\mat{X}\vect{w} = \vect{0}}\) only for \({\vect{w} = \vect{0}}\).
Exercises
1.★ For which numbers \(y\) is \(\begin{bmatrix} 2 & 6 \\ 6 & y \end{bmatrix}\) positive definite? What happens at the borderline value?
Show solution
\({a = 2 > 0}\), so it is definite exactly when \({2y - 36 > 0}\), that is, \({y > 18}\). At \({y = 18}\) the determinant is zero and the form is \({2x^2 + 12xy + 18y^2 = 2(x + 3y)^2}\), which is zero along the line \({x = -3y}\): semidefinite, with eigenvalues 0 and 20.
2.★★ Find the pivots and leading minors of \(\begin{bmatrix} 4 & 2 & 2 \\ 2 & 5 & 3 \\ 2 & 3 & 6 \end{bmatrix}\) and decide whether it is positive definite.
Show solution
Subtract \(\tfrac12\) of row 1 from rows 2 and 3: they become \({(0, 4, 2)}\) and \({(0, 2, 5)}\). Subtract \(\tfrac12\) of the new row 2 from row 3: \({(0, 0, 4)}\). The pivots are 4, 4 and 4, and the leading minors are their running products 4, 16 and 64. All positive, so the matrix is positive definite. Check the last minor by cofactors: \({4(30 - 9) - 2(12 - 6) + 2(6 - 10)} = {84 - 12 - 8} = 64\).
3.★★★ Let \(\mat{X}\) be any matrix and \({\alpha > 0}\). Prove that every eigenvalue of \({\mat{X}\T\mat{X} + \alpha\mat{I}}\) is at least \(\alpha\).
Show solution
Let \(\vect{v}\) be a unit eigenvector with eigenvalue \(\lambda\). Then \({\vect{v}\T(\mat{X}\T\mat{X} + \alpha\mat{I})\vect{v}} = {\lambda\,\vect{v}\T\vect{v}} = \lambda\). The left side equals \({\norm{\mat{X}\vect{v}}^2 + \alpha\norm{\vect{v}}^2} = {\norm{\mat{X}\vect{v}}^2 + \alpha}\), which is at least \(\alpha\). So \({\lambda \geq \alpha}\). In the code above, \({\alpha = 1}\) lifted the eigenvalues 0 and 70 to 1 and 71.
Summary card
Positive definite means the form is a bowl: positive in every direction.
M03.4.2-F1
Definite, semidefinite, indefinite.
Definition.\({\vect{x}\T\mat{S}\vect{x} > 0}\) for all \({\vect{x} \neq \vect{0}}\).
Tests. All \({\lambda_i > 0}\); all pivots \({> 0}\); all leading minors \({> 0}\).
Ridge.\({\mat{X}\T\mat{X} + \alpha\mat{I}}\) is definite for \({\alpha > 0}\).
Sources and next steps
Sources. Each was opened and checked against this lesson on 15 November 2026.
Gilbert Strang, Positive definite matrices and minima, MIT OpenCourseWare 18.06SC, lecture summary. It lists the eigenvalue, determinant and pivot tests for a 2 by 2 matrix, works the matrix with rows \({(2, 6)}\) and \({(6, y)}\) with its borderline \({y = 18}\), completes the square \({2(x + 3y)^2 + 2y^2}\) and says that “the numbers multiplied by the squares are the pivots”.
David Austin, Understanding Linear Algebra, chapter 7, section “Quadratic forms”: the definitions of positive definite, positive semidefinite and indefinite, and their classification by the signs of the eigenvalues.
Marc Peter Deisenroth, A. Aldo Faisal and Cheng Soon Ong, Mathematics for Machine Learning. Definition 3.4 defines symmetric positive definite and semidefinite matrices; Theorem 4.14 and the remark after it state that \({\mat{A}\T\mat{A}}\) is semidefinite, and definite when \(\mat{A}\) has full column rank.
NumPy reference, numpy.linalg.eigvalsh: the eigenvalues of a symmetric matrix, in ascending order.