Numerical methods

Numerical methods refer to a broad field of mathematics focused on finding approximate solutions via an iterative update of estimates, instead of analytically/symbolically finding the exact correct solution.

This is useful because we often must find solutions for many problems on computers, which work best in an iterative way.

Basics

Computers are only able to store a subset of the real numbers. This means that we must incur some approximation error when performing computations. If our algorithms don’t minimise the accumulation of rounding errors, they could fail in unexpected ways.

Underflow is when numbers near 0 are rounded to 0. Overflow is when we approximate large numbers with $\pm \infty$. This implication implies that basic operations (like addition, multiplication) may not necessarily be associative, i.e., $(a+b)+c\ne (b+c)+a$. Algorithms that don’t account for this are numerically unstable.

Good numerical algorithms also consider the computing platform they run on. Memory layout, processor architecture, and parallelisation opportunities determine the speed and performance of algorithms.

Sub-pages

• Optimisation applications
  • Gradient descent
• Calculus and ODEs
  • Numerical integration
  • Runge-Kutta methods
    • Euler’s method