Central limit theorem

The central limit theorem (CLT) is a foundational theorem in statistics and probability theory. It states that for a sum of $n$ independent, identically distributed (iid) random variables with finite mean $\mu$ and finite variance ${\sigma }^2$:

$$S_n=X_1+X_2+\cdots +X_n$$

The normalised sum’s CDF is approximately normal, regardless of what population the sample was drawn from (this is why the Gaussian shows up everywhere in nature). The random variables $X_j$ can have any distribution as long as they have a finite mean and finite variance.

$$\frac{X_1+\cdots +X_n}{\sqrt{n}}\to \mathcal{N}(0,{\sigma }^2)$$

Formal statement

Let $S_n$ be the sum of $n$ iid random variables with finite mean $\mu$ and finite variance ${\sigma }^2$. Let $Z_n$ be the zero mean, unit variance random variable:

$$Z_n=\frac{S_n-n\mu }{\sigma \sqrt{n}}$$

Then:

$$\underset{n\to \infty }{\lim }P(Z_n\le z)=\frac{1}{\sqrt{2\pi }}{\int }_{\infty }^z\exp \left(-\frac{x^2}{2}\right)dx$$

Proof

https://www.cs.toronto.edu/~yuvalf/CLT.pdf two proofs