Zipf distribution

The Zipf distribution is a probability distribution used primarily for text frequency, where it models the frequency of words in a large body of text being proportional to their rank (i.e., its frequency rank).

The probability mass function is given by:

$$p_X(k)=\frac{1}{c_L}\frac{1}{k}$$

where $L$ is the number of distinct words, $k\in [1,L]$, $S_X\in [1,L]$ is the number of occurrences of the word, and $c_L$ is a normalisation constant (the $L$th harmonic mean), given by:

$$c_L=\sum _{j=1}^L\frac{1}{j}=1+\frac{1}{2}+\frac{1}{3}+\cdots +\frac{1}{L}$$

The Zipf random variable has a property such that few outcomes (words) occur frequently, but most outcomes occur rarely. It finds use in studies on the Internet and interconnectivity.

Computations

The expected value is given by:

$$E(X)=\sum _{j=1}^{L}j{p}_X(j)=\sum _{j=1}^{L}j\frac{1}{c_{L}j}=\frac{L}{c_L}$$

The second moment is given by:

$$E(X^2)=\frac{L(L+1)}{2c_L}$$

And variance:

$$\text{Var}(X)=\frac{L(L+1)}{2c_L}-\frac{L^2}{c_L^2}$$