Moore-Penrose pseudoinverse

The pseudoinverse of a matrix $A$ is defined as:

$$A^{+}=\underset{\alpha \to 0}{\lim }(A^{T}A+\alpha I{)}^{-1}{A}^T$$

Why do we even care? Matrix inversion is only defined for square matrices. If $A$ isn’t square, we run the risk of finding infinitely many solutions or none at all. Pseudoinverses allow us to approximate the best inverse solution.

The Moore-Penrose pseudoinverse is a practical algorithm for computing a pseudoinverse matrix via singular value decomposition, where:

$$A^{+}=V{D}^{+}{U}^T$$

where $U,D,V$ are the SVD of $A$. The pseudoinverse $D^{+}$ is computed by taking the reciprocal of its elements then transposing it.

If we have infinitely many solutions, MP gives us a solution $\vec{x}=A^{+}\vec{y}$ that minimises the Euclidean norm. If we have no solution, MP gives us the $x$ for which $A\vec{x}$ is as close to possible to $\vec{y}$ in terms of the norm of $A\vec{x}-\vec{y}$.