Hessian matrix

The Hessian matrix is a square matrix of second-order partial derivatives of a scalar-valued function. It’s used in the second-derivative test for multivariable functions.

$$H_f=\left[\begin{array}{cccc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_1\partial x_n}\\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}& \cdots & \frac{{\partial }^{2}f}{\partial x_2\partial x_n}\\ ⋮& ⋮& \ddots & ⋮\\ \frac{{\partial }^{2}f}{\partial x_n\partial x_1}& \frac{{\partial }^{2}f}{\partial x_n\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_n^2}\end{array}\right]$$

The Hessian is the Jacobian matrix of the gradient.

One key idea: if the second-partial derivatives are continuous at a point, the operators are commutative. This implies that the Hessian is symmetric at such points. Because it is real and symmetric, then we can decompose it into a set of real eigenvalues and an orthogonal basis of eigenvectors.

This eigendecomposition allows us to generalise the second-derivative test to multiple dimensions.