Function extrema

Functions of a single or multiple real variables have important extrema points that characterise the function, called maxima and minima. Formally, let’s set up some definitions. Let $I\subseteq {\mathbb{R}}^n$ be an interval and $f:I\to {\mathbb{R}}^n$ be a function. A point $p_0\in I$ is:

• A local maximum if there exists a neighbourhood $\mathcal{U}$ of $p_0$ such that $f(p)\le f(p_0)$ for all points in the neighbourhood.
  • A strict local maximum if the neighbourhood is such that $f(p)<f(p_0)$ for all $x\in \mathcal{U}\backslash \{p_0\}$.
• A global maximum if $f(p)\le f(p_0)$ for every $p\in {\mathbb{R}}^n$, i.e., it is the highest peak
• A local minimum if there exists a neighbourhood of $p_0$ such that $f(p)\ge f(p_0)$ for all points in the neighbourhood.
• A global minimum if $f(p)\ge f(p_0)$ for every $p\in {\mathbb{R}}^n$, i.e., it is the lowest trough.

Finding the extrema and other related characteristics of a function forms a primitive but usable example of mathematical programming (optimisation).

Definitions and theorems

We should set up a few more definitions. For an interval $I\subseteq \mathbb{R}$, a function $f:I\to \mathbb{R}$ is:

• Monotonically increasing if for every $x_1,x_2\in I$ with $x_1<x_2$, then $f(x_1)\le f(x_2)$.
• Monotonically decreasing if for every $x_1,x_2\in I$ with $x_1<x_2$, then $f(x_1)\ge f(x_2)$.
• Constant if there exists $\alpha \in \mathbb{R}$ such that $f(x)=\alpha \forall x\in I$.

If $f^{\prime }$ is increasing on $I$, then $f$ is concave up. If $f^{\prime }$ is decreasing on $I$, then $f$ is concave down. Then, if $f$ is continuous and changes concavity at $x_0$, then it has an inflection point.

The extreme value theorem tells us that a continuous function on an interval $[a,b]$ has an absolute maximum and minimum value on that interval. Intuitively this makes a ton of sense.

Single-variable case

By theorem, the first derivative test tells us that if $f(x)$ is differentiable at a point $x_0$, and $x_0$ is a local extrema for $f$, then $f^{\prime }(x_0)=0$. If $f$ is twice differentiable at $x_0$, then $f^{\prime \prime }(x_0)\le 0$ if $x_0$ is a local maximum and $f^{\prime \prime }(x_0)\ge 0$ if $x_0$ is a local minimum.

By the second derivative test, if $f^{\prime \prime }>0$, then it is concave up. If $f^{\prime \prime }<0$, then it is concave down on the interval. If $f^{\prime \prime }(c)>0$, then $f$ has a local minimum at $c$. If $f^{\prime \prime }(c)<0$, then it has a local maximum at $c$.

Multivariable case

In the multivariable case, if the first partial derivatives are equal to 0, we have a critical point.

Lagrange multipliers are a common method for solving multivariable optimisation problems that are constrained.