Skip to main content

Second partial derivatives and the Hessian

Differentiate every partial derivative again and you get n squared second partial derivatives. Stack them into the Hessian, the Jacobian of the gradient, see why it is symmetric, meet the function where it is not, and compute it by hand, by finite differences and with SciPy.
M05.5.1L2 Intermediate25 min
In review

The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

Before this lesson: M05.2 Partial derivatives and the gradient M05.4 Jacobians and the multivariable chain rule M03.3 Symmetric matrices and the spectral theorem

What you will be able to do
  • Compute every second partial derivative of a function of several inputs and arrange them in the Hessian matrix.
  • Explain the Hessian as the Jacobian of the gradient: column j says how the gradient changes per unit step along input j.
  • State Schwarz’s theorem on mixed partial derivatives, follow its proof, and use Peano’s function to show why its hypothesis matters.
  • Compute a Hessian by central differences of the gradient and check it with scipy.differentiate.hessian.

The question

At the origin the gradient of the running loss is \({(-23.5, -8)}\): it says which way is uphill and how steep the climb is. Take one step and the gradient is different. How fast does the gradient itself change, and does it change at the same rate in every direction? The answer is a matrix of second derivatives, and every method that is smarter than plain gradient descent, from Newton’s method to the rule that caps the learning rate, is built on it.

Intuition

The gradient is an arrow attached to every point. Move the point and the arrow turns and stretches. The quickest way to see how is to take one small step along each input and compare the arrows.

Left: a contour map of the running bowl x squared plus 2 y squared with blue oval contour lines. An orange point at (1, 1) carries a green gradient arrow labelled (2, 4). Dashed grey steps of 0.5 lead right to a hollow point at (1.5, 1), whose green arrow is labelled (3, 4), and up to a hollow point at (1, 1.5), whose green arrow is labelled (2, 6). Right: the changes written out: step 0.5 in x changes the gradient by (1, 0), step 0.5 in y changes it by (0, 2). Per unit of step these are the columns of the matrix H with rows 2, 0 and 0, 4, the x column shaded green and the y column paler green.
M05.5.1-F1

Step from \({(1, 1)}\) along \(x\) and the gradient of the running bowl changes only in its first entry; step along \(y\) and only the second changes, twice as fast. Those two changes per unit of step, side by side, are the Hessian.

For the running bowl \({f(x, y) = x^2 + 2y^2}\) the gradient is \({(2x, 4y)}\). A step of 0.5 along \(x\) moves it from \({(2, 4)}\) to \({(3, 4)}\), a change of \({(1, 0)}\), or \({(2, 0)}\) per unit of step. A step of 0.5 along \(y\) moves it to \({(2, 6)}\), a change of \({(0, 4)}\) per unit. Stand the two changes up as columns and you have a \({2 \times 2}\) matrix, the Hessian. It is to the gradient what the derivative is to a function: its rate of change.

The two columns are not equal. Along \(y\) the gradient grows twice as fast, because the bowl curves up twice as sharply in that direction. That is the first hint of what the Hessian measures: curvature, and how it differs from one direction to another.

Build it up

Step 1. A partial derivative is a function too. For the running loss

\[ \begin{aligned} L(w, b) &= 7.5w^2 + 5wb + b^2 \\ &\quad - 23.5w - 8b + 18.5 , \end{aligned} \]

the two partial derivatives are \({\pd{L}{w} = 15w + 5b - 23.5}\) and \({\pd{L}{b} = 5w + 2b - 8}\). Each is a new function of \(w\) and \(b\), with slopes of its own.

Step 2. Differentiate each one again, along each input. The slope of \({\pd{L}{w}}\) as \(w\) moves is 15. Its slope as \(b\) moves is 5. A second partial derivative is the slope of a slope.

Two line graphs side by side. Left, titled d L by d w as w moves: a green straight line, the partial derivative 15 w minus 23.5 with b held at 0, starting at an orange dot at minus 23.5 on the vertical axis. A run-and-rise triangle under it reads run 0.6, rise 9: slope 15. Right, titled d L by d w as b moves: a flatter green line, minus 23.5 plus 5 b with w held at 0, with a triangle reading run 0.6, rise 3: slope 5. A line above both reads: slopes of a slope, the second derivative in w twice is 15 and the mixed one is 5.
M05.5.1-F2

The partial derivative \({\pd{L}{w}}\) is a straight line in each input. Its slope along \(w\) is 15 and its slope along \(b\) is 5: two second partial derivatives of the running loss.

Step 3. Name the four results. Differentiating \({\pd{L}{b}}\) the same way gives 5 along \(w\) and 2 along \(b\). Write \({L_{ww} = 15}\), \({L_{wb} = 5}\), \({L_{bw} = 5}\) and \({L_{bb} = 2}\), where the subscripts list the inputs in the order you differentiate: \(L_{wb}\) means “first along \(w\), then along \(b\)”.

Step 4. Stack them. Put the slopes of \({\pd{L}{w}}\) in row 1 and the slopes of \({\pd{L}{b}}\) in row 2:

\[ \mat{H} = \begin{bmatrix} L_{ww} & L_{wb} \\ L_{bw} & L_{bb} \end{bmatrix} = \begin{bmatrix} 15 & 5 \\ 5 & 2 \end{bmatrix} . \]

The diagonal entries say how sharply the loss curves along each input on its own. The off-diagonal 5 comes from the term \({5wb}\): it says how the slope in \(w\) changes when \(b\) moves. The running bowl has no \({xy}\) term, so its Hessian \({\begin{bmatrix} 2 & 0 \\ 0 & 4 \end{bmatrix}}\) has zeros off the diagonal.

The formal statement

NoteDefinition: second partial derivatives and the Hessian

Let \({f : \R^n \to \R}\) have partial derivatives that themselves have partial derivatives at \(\vect{a}\). The second partial derivative along \(x_i\) and then \(x_j\) is

\[ \frac{\partial^2 f}{\partial x_j\, \partial x_i}(\vect{a}) = \frac{\partial}{\partial x_j}\!\left(\pd{f}{x_i}\right)\!(\vect{a}) . \]

The Hessian \(\mat{H}_f(\vect{a})\), also written \({\nabla^2 f(\vect{a})}\), is the \({n \times n}\) matrix whose entry in row \(i\) and column \(j\) is that number.

Row \(i\) of the Hessian holds the partial derivatives of the single function \({\pd{f}{x_i}}\), which is output \(i\) of the gradient map \({\vect{x} \mapsto \nabla f(\vect{x})}\). In the numerator layout of M05.4, row \(i\) of a Jacobian is the transposed gradient of output \(i\). So

\[ \mat{H}_f(\vect{a}) = \mat{J}_{\nabla f}(\vect{a}) : \]

the Hessian is the Jacobian of the gradient. The linear approximation of M05.4.1, applied to the gradient map, then says what the Hessian does:

\[ \nabla f(\vect{a} + \vect{h}) \approx \nabla f(\vect{a}) + \mat{H}_f(\vect{a})\, \vect{h} . \]

Column \(j\) is \({\mat{H}\vect{e}_j}\), the change of the gradient per unit step along input \(j\): exactly the picture of the anchor figure. For the bowl the gradient is linear, so the approximation is exact: \({(2, 4) + \mat{H}\,(0.5, 0) = (3, 4)}\).

Are the two mixed entries always equal? In every example so far, yes. The theorem below says when that is guaranteed.

NoteTheorem: symmetry of second derivatives (Schwarz, also called Clairaut)

If all the second partial derivatives of \(f\) exist and are continuous on an open ball around \(\vect{a}\), then for every \(i\) and \(j\)

\[ \frac{\partial^2 f}{\partial x_j\, \partial x_i}(\vect{a}) = \frac{\partial^2 f}{\partial x_i\, \partial x_j}(\vect{a}) , \]

so the Hessian is a symmetric matrix.

Symmetry is what makes the Hessian so useful. By the spectral theorem (M03.3) a symmetric matrix has real eigenvalues and perpendicular eigenvectors, and the next two lessons read curvature off exactly those.

Go deeper: the proof, by a double difference quotient

Take two inputs, a point \({(a, b)}\), and small steps \({h, k \neq 0}\). Form the double difference

\[ \begin{aligned} \Delta &= f(a + h, b + k) - f(a + h, b) \\ &\quad - f(a, b + k) + f(a, b) . \end{aligned} \]

First grouping. Let \({g(x) = f(x, b + k) - f(x, b)}\), so that \({\Delta = g(a + h) - g(a)}\). By the mean value theorem there is a \(\xi\) between \(a\) and \({a + h}\) with

\[ \begin{aligned} \Delta &= h\, g'(\xi) \\ &= h \big( f_x(\xi, b + k) - f_x(\xi, b) \big) . \end{aligned} \]

The bracket is a change of the function \({y \mapsto f_x(\xi, y)}\), so the mean value theorem again gives an \(\eta\) between \(b\) and \({b + k}\) with \({\Delta = hk\, f_{xy}(\xi, \eta)}\).

Second grouping. Let \({p(y) = f(a + h, y) - f(a, y)}\), so that \({\Delta = p(b + k) - p(b)}\). The same two steps, in the other order, give points \(\xi'\) and \(\eta'\) in the same rectangle with \({\Delta = hk\, f_{yx}(\xi', \eta')}\).

Conclusion. Divide both by \({hk}\): \({f_{xy}(\xi, \eta) = f_{yx}(\xi', \eta')}\). As \(h\) and \(k\) shrink to 0 both points close in on \({(a, b)}\), and because both mixed partial derivatives are continuous there, the two sides tend to \({f_{xy}(a, b)}\) and \({f_{yx}(a, b)}\). They are equal. For \(n\) inputs apply the argument to any two of them with the rest held fixed. \(\square\)

When the hypothesis fails. Peano’s function is

\[ f(x, y) = \frac{xy\,(x^2 - y^2)}{x^2 + y^2} , \]

with \({f(0, 0) = 0}\).

Along the \(y\)-axis, for \({y \neq 0}\), the slope in \(x\) is

\[ \begin{aligned} f_x(0, y) &= \lim_{h \to 0} \frac{f(h, y) - f(0, y)}{h} \\ &= \lim_{h \to 0} \frac{y\,(h^2 - y^2)}{h^2 + y^2} = -y , \end{aligned} \]

because \({f(0, y) = 0}\). At \({y = 0}\) the same holds, since \({f(h, 0) = 0}\). In the same way \({f_y(x, 0) = x}\) along the \(x\)-axis.

Two line graphs. Left, titled f sub x of 0 comma y equals minus y: a green line falling from (minus 1, 1) to (1, minus 1) through an orange dot at the origin, labelled slope minus 1, plotted against y on the line x equals 0. Right, titled f sub y of x comma 0 equals x: a green line rising from (minus 1, minus 1) to (1, 1) through an orange dot at the origin, labelled slope plus 1, plotted against x on the line y equals 0. A sentence above reads: so f sub x y at the origin is minus 1 but f sub y x is plus 1, the order matters here.
M05.5.1-F5

Peano’s function. The slope in \(x\), measured up the \(y\)-axis, falls with slope \(-1\); the slope in \(y\), measured along the \(x\)-axis, rises with slope \(+1\). The two mixed partial derivatives at the origin differ.

Differentiate once more: \({f_{xy}(0, 0) = \tfrac{d}{dy}(-y) = -1}\) but \({f_{yx}(0, 0) = \tfrac{d}{dx}(x) = 1}\). No contradiction: away from the origin both mixed partial derivatives equal

\[ \frac{(x^2 - y^2)(x^4 + 10x^2y^2 + y^4)}{(x^2 + y^2)^3} , \]

which is 1 along the \(x\)-axis and \(-1\) along the \(y\)-axis, so it is not continuous at the origin.

Worked example

The Hessian of Rosenbrock’s function \({f(x, y) = (1 - x)^2 + 100\,(y - x^2)^2}\) at its minimum \({(1, 1)}\).

First derivatives. The chain rule on each square gives

\[ \begin{aligned} f_x &= -2(1 - x) - 400x\,(y - x^2) , \\ f_y &= 200\,(y - x^2) . \end{aligned} \]

Second derivatives. Differentiate \(f_x\) along \(x\) with the product rule on \({-400x\,(y - x^2)}\):

\[ \begin{aligned} f_{xx} &= 2 - 400\,(y - x^2) \\ &\quad - 400x\,(-2x) \\ &= 2 - 400y + 1200x^2 . \end{aligned} \]

Along \(y\) only the last term of \(f_x\) depends on \(y\), so \({f_{xy} = -400x}\). From \(f_y\): \({f_{yx} = 200 \cdot (-2x) = -400x}\) and \({f_{yy} = 200}\). The mixed pair agree, as Schwarz’s theorem promises for a polynomial.

A tree of boxes. At the top a blue box holds Rosenbrock's function, f equals (1 minus x) squared plus 100 (y minus x squared) squared. Arrows marked d by d x and d by d y lead to two green boxes, f sub x equals minus 2 (1 minus x) minus 400 x (y minus x squared), and f sub y equals 200 (y minus x squared). Arrows marked x and y lead from each to four grey boxes: f sub x x equals 2 minus 400 y plus 1200 x squared, f sub x y equals minus 400 x, f sub y x equals minus 400 x, f sub y y equals 200. Below, at (1, 1): f sub x x equals 2 minus 400 plus 1200 equals 802, f sub x y equals f sub y x equals minus 400, f sub y y equals 200, and the matrix H sub f of (1, 1) with rows 802, minus 400 and minus 400, 200, with the note row 1 from f sub x, row 2 from f sub y.
M05.5.1-F3

The worked example as a trace: differentiate once, differentiate each result again along each input, then evaluate at \({(1, 1)}\).

Evaluate at \({(1, 1)}\). \({f_{xx} = 2 - 400 + 1200 = 802}\), \({f_{xy} = f_{yx} = -400}\) and \({f_{yy} = 200}\):

\[ \mat{H}_f(1, 1) = \begin{bmatrix} 802 & -400 \\ -400 & 200 \end{bmatrix} . \]

A second point. At the origin the same formulas give \({f_{xx} = 2}\), \({f_{xy} = 0}\) and \({f_{yy} = 200}\). Unlike the bowl and the running loss, whose Hessians are the same everywhere, Rosenbrock’s Hessian changes from point to point, because the function is not a quadratic.

Code

From scratch: column \(j\) of the Hessian is the change of the gradient along input \(j\), measured by a central difference. That costs \({2n}\) gradient evaluations.

import numpy as np

def rosen(p):
    x, y = p
    return (1 - x) ** 2 + 100 * (y - x ** 2) ** 2

def rosen_grad(p):
    x, y = p
    return np.array([-2 * (1 - x) - 400 * x * (y - x ** 2), 200 * (y - x ** 2)])

def hessian_from_gradient(grad, a, h=1e-5):
    """Column j: (grad(a + h e_j) - grad(a - h e_j)) / 2h."""
    a = np.asarray(a, dtype=float)
    n = a.size
    H = np.empty((n, n))
    for j in range(n):
        e = np.zeros(n)
        e[j] = h
        H[:, j] = (grad(a + e) - grad(a - e)) / (2 * h)
    return H

H_fd = hessian_from_gradient(rosen_grad, [1.0, 1.0])
print(np.round(H_fd, 4))
print("asymmetry:", abs(H_fd[0, 1] - H_fd[1, 0]))

loss_grad = lambda p: np.array([15 * p[0] + 5 * p[1] - 23.5, 5 * p[0] + 2 * p[1] - 8])
print(np.round(hessian_from_gradient(loss_grad, [0.0, 0.0]), 6))
[[ 802. -400.]
 [-400.  200.]]
asymmetry: 0.0
[[15.  5.]
 [ 5.  2.]]

A finite-difference Hessian is symmetric only up to rounding, so the two mixed entries can differ in their last digits. Now the library. scipy.differentiate.hessian wants a function that takes an array whose first axis holds the inputs, and it returns an object whose ddf attribute is the Hessian.

from scipy.differentiate import hessian

res = hessian(rosen, np.array([1.0, 1.0]))
print(res.ddf)
print("converged:", res.success.all())
[[ 802. -400.]
 [-400.  200.]]
converged: True

Finally, Peano’s function: differentiate in the two orders at the origin, with a tiny inner step inside a larger outer one.

def peano(x, y):
    return 0.0 if x == 0 and y == 0 else x * y * (x ** 2 - y ** 2) / (x ** 2 + y ** 2)

h, k = 1e-7, 1e-3                                       # inner step, outer step
fx = lambda x, y: (peano(x + h, y) - peano(x - h, y)) / (2 * h)
fy = lambda x, y: (peano(x, y + h) - peano(x, y - h)) / (2 * h)
f_xy = (fx(0, k) - fx(0, -k)) / (2 * k)                 # first along x, then along y
f_yx = (fy(k, 0) - fy(-k, 0)) / (2 * k)                 # first along y, then along x
print(round(f_xy, 4), round(f_yx, 4))
-1.0 1.0

Explore

This module’s interactive, the curvature viewer, lives in lesson M05.5.2. Pick Rosenbrock there and drag the point from the origin towards \({(1, 1)}\): the Hessian in the read-out changes at every step, while for the running bowl and the saddle it never does.

Pitfalls

Two contour maps of the running loss over w from 0.2 to 2.6 and b from minus 1.6 to 2.6, each with solid reddish purple tilted ellipses around a black star at (1.4, 0.5). Left, marked with a cross and titled Diagonal only: dashed vermillion ellipses built from the matrix with rows 15, 0 and 0, 2 stand upright and do not follow the tilted solid ones. Right, marked with a tick and titled Full Hessian: dashed vermillion ellipses built from the matrix with rows 15, 5 and 5, 2 lie exactly on the solid ones.
M05.5.1-F4

Wrong beside right. A quadratic model built from the diagonal entries alone has upright contour lines; with the off-diagonal 5 it lies exactly on the running loss.

“The curvature is just the two second derivatives along the axes.”

The diagonal entries describe the function along the coordinate axes only. The off-diagonal entries tilt the picture: without them the contour lines of the running loss would stand upright, as on the left of the figure, and the valley would point the wrong way. In a model with many weights most of the interesting curvature is off the diagonal, between weights that work together.

“The mixed partial derivatives are always equal, so I never need to check.”

They are equal when they are continuous, which covers every polynomial, the sigmoid, \(\tanh\) and the usual losses. Peano’s function shows that the hypothesis is not decoration. Networks with ReLU units have kinks where second derivatives do not exist at all, so a Hessian there is a statement about the smooth pieces on either side.

“numpy.linalg.eigvalsh will tell me the eigenvalues of my numerical Hessian.”

It will, but it reads only the lower triangle of the matrix you pass. A finite-difference Hessian that is slightly asymmetric, or a matrix with a bug in its upper half, is silently replaced by a symmetric one. Symmetrise first, with \({(\mat{H} + \mat{H}\T)/2}\), and check that the asymmetry you removed was small.

Check yourself

Recall

1. What is in row \(i\), column \(j\) of the Hessian of \({f : \R^n \to \R}\)?

Show answer

The second partial derivative \({\frac{\partial}{\partial x_j}\big(\pd{f}{x_i}\big)}\): differentiate along \(x_i\), then along \(x_j\).

2. The Hessian is the Jacobian of which map, and what does \({\mat{H}\vect{h}}\) approximate?

Show answer

Of the gradient map \({\vect{x} \mapsto \nabla f(\vect{x})}\). \({\mat{H}\vect{h}}\) approximates the change of the gradient, \({\nabla f(\vect{a} + \vect{h}) - \nabla f(\vect{a})}\).

3. Under what condition is the Hessian symmetric, and what is the theorem called?

Show answer

When the second partial derivatives are continuous near the point: Schwarz’s theorem, also called Clairaut’s theorem.

4. What are the Hessians of the running bowl and of the running loss?

Show answer

\({\begin{bmatrix} 2 & 0 \\ 0 & 4 \end{bmatrix}}\) and \({\begin{bmatrix} 15 & 5 \\ 5 & 2 \end{bmatrix}}\), the same at every point because both functions are quadratic.

5. In the running loss, which term produces the off-diagonal 5, and what does that entry mean?

Show answer

The term \({5wb}\). The entry says how fast the slope in \(w\) changes when \(b\) moves, and equally how fast the slope in \(b\) changes when \(w\) moves.

Exercises

1. ★ Find the Hessian of \({f(x, y) = x^3 y + 2y^2 - xy}\) at \({(1, 2)}\).

Show solution

\({f_x = 3x^2 y - y}\) and \({f_y = x^3 + 4y - x}\). Then \({f_{xx} = 6xy}\), \({f_{xy} = 3x^2 - 1}\), \({f_{yx} = 3x^2 - 1}\) and \({f_{yy} = 4}\). At \({(1, 2)}\):

\[ \mat{H} = \begin{bmatrix} 12 & 2 \\ 2 & 4 \end{bmatrix} . \]

2. ★★ Let \({\mat{A} = \begin{bmatrix} 1 & 2 \\ 0 & 3 \end{bmatrix}}\), which is not symmetric, and \({f(\vect{x}) = \vect{x}\T \mat{A} \vect{x}}\) for \({\vect{x} = (x, y)}\). Write \(f\) out, find its Hessian, and express it in terms of \(\mat{A}\).

Show solution

\({\vect{x}\T \mat{A} \vect{x} = x^2 + 2xy + 3y^2}\): the entry 2 above the diagonal and the 0 below it add up to the single coefficient of \({xy}\). Then \({f_{xx} = 2}\), \({f_{xy} = f_{yx} = 2}\) and \({f_{yy} = 6}\), so

\[ \mat{H} = \begin{bmatrix} 2 & 2 \\ 2 & 6 \end{bmatrix} = \mat{A} + \mat{A}\T . \]

The Hessian is symmetric even though \(\mat{A}\) is not; for a symmetric \(\mat{A}\) it is \({2\mat{A}}\).

3. ★★★ For Rosenbrock’s function, compute the double difference \(\Delta\) of the proof at \({(1, 1)}\) with \({h = k = 0.1}\), and divide by \({hk}\). You get \(-420\), not \({f_{xy}(1, 1) = -400}\). Show that \({\Delta / (hk)}\) is the average of \(f_{xy}\) over the square \({[1, 1.1] \times [1, 1.1]}\), and use this to explain \(-420\) exactly.

Show solution

The four values are

\[ \begin{aligned} f(1.1, 1.1) &= 0.01 + 100\,(-0.11)^2 \\ &= 1.22 , \\ f(1.1, 1) &= 0.01 + 100\,(-0.21)^2 \\ &= 4.42 , \\ f(1, 1.1) &= 100\,(0.1)^2 = 1 , \\ f(1, 1) &= 0 . \end{aligned} \]

So \(\Delta = 1.22 - 4.42 - 1 + 0 = -4.2\) and \({\Delta / 0.01} = -420\).

By the fundamental theorem of calculus along \(y\), \({f_x(x, b + k) - f_x(x, b)}\) is the integral of \({f_{xy}(x, y)}\) over \({y}\) from \(b\) to \({b + k}\). Integrating that along \(x\) from \(a\) to \({a + h}\) turns the left side into \(\Delta\), because \(f_x\) integrates back to \(f\). So \(\Delta\) is the double integral of \(f_{xy}\) over the square, and \({\Delta / (hk)}\) is its average there. Here \({f_{xy} = -400x}\), whose average over the square is \({-400 \times 1.05 = -420}\), since \(x\) averages 1.05. As the square shrinks the average tends to \({f_{xy}(1, 1) = -400}\).

Summary card

The Hessian is the rate of change of the gradient.

The anchor figure again: on the running bowl's contour map, gradient arrows at (1, 1), (1.5, 1) and (1, 1.5), and beside them the Hessian with rows 2, 0 and 0, 4, whose columns are the changes of the gradient per unit step in x and in y.
M05.5.1-F1

Columns: how the gradient moves.

  • Entries. \({\mat{H}_{ij} = \frac{\partial}{\partial x_j}\big(\pd{f}{x_i}\big)}\), an \({n \times n}\) matrix.
  • Meaning. \({\mat{H}_f = \mat{J}_{\nabla f}}\), so \({\nabla f(\vect{a} + \vect{h}) \approx \nabla f(\vect{a}) + \mat{H}\vect{h}}\).
  • Symmetry. Continuous second partials give \({\mat{H} = \mat{H}\T}\) (Schwarz).
  • Running examples. Bowl \({\operatorname{diag}(2, 4)}\); loss \({[[15, 5], [5, 2]]}\); Rosenbrock at \({(1, 1)}\): \({[[802, -400], [-400, 200]]}\).

Sources and next steps

Sources. Each was opened and checked against this lesson on 15 November 2026.

  • Gilbert Strang and Edwin Herman, Calculus Volume 3, Section 4.3, Partial derivatives, OpenStax. Defines the higher-order partial derivatives with the subscript order used here (\({f_{xy}}\) is differentiated first in \(x\)) and states Theorem 4.5, the equality of mixed partial derivatives (Clairaut’s theorem), under continuity on an open disc.
  • Marc Peter Deisenroth, A. Aldo Faisal and Cheng Soon Ong, Mathematics for Machine Learning, Section 5.7. Equation (5.146) states that the mixed partials of a twice continuously differentiable function agree and (5.147) defines the symmetric Hessian.
  • Alan Edelman and Steven G. Johnson, Matrix Calculus for Machine Learning and Beyond, MIT course notes, Section 12.1: the Hessian as the Jacobian of \({\nabla f}\), and \({\mat{H}\,d\vect{x}}\) as the first-order change of the gradient.
  • Wikipedia, Symmetry of second derivatives: Schwarz’s theorem, the proof by the mean value theorem applied to a double difference, and Peano’s counterexample with mixed partials \(-1\) and \(1\) at the origin.
  • SciPy reference, scipy.differentiate.hessian (the input axis convention and the ddf attribute), and NumPy reference, numpy.linalg.eigvalsh (only one triangle, the lower by default, is used).

Next. M05.5.2 The second-order Taylor model puts the Hessian to work: value, gradient and Hessian together give the best quadratic approximation of a function near a point.

This lesson unlocks

M05.5.2 The second-order Taylor model

Your progress is saved in this browser only. No account, no tracking.
Back to top