Probability distribution

The probability distribution of a random variable $X$ is the function that describes the probabilities for the set of possible values. We define two main types of distributions:

• The probability mass function (PMF) of a discrete random variable is the function $p(x)=P(X=x)$ that describes the probabilities for the set of possible values.
• The probability density function (PDF) describes a continuous random variable’s $X$ probability (since its probabilities are given by the area under the curve). We define the PDF as the derivative of the CDF (at places where it’s differentiable):

$$f_X(x)=\frac{d{F}_X(t)}{dx}$$ $${\int }_{-\infty }^{\infty }{f}_X(x)=1$$

The cumulative distribution function (CDF) specifies the probability that a random variable is less than or equal to a given value:

$$F_X(x)=P(X\le x)={\int }_{-\infty }^{x}f(t)dt$$

where $X$ is a random variable, and $x$ is an index. From the fundamental axioms of probability, $F_X(x)$ is monotonically increasing (since it accumulates the probabilities).

We can rigorously build up the theory behind random variables and probability distributions with measure theory. Special probability distributions are determined by special random variables with particular properties.

Fundamentals

For a probability density function:

$$P(a\le X\le b)=P(x\le b)-P(x\le a)$$ $$P(a\le x\le b)={\int }_{-\infty }^b{f}_x(u)du-{\int }_{-\infty }^a{f}_x(u)du={\int }_a^b{f}_x(u)du$$

We define the conditional CDF of $X$ given $A$ as:

$$F_X(x|A)=\frac{P(\{X\le x\}\cap A)}{P(A)}\text{if}P(A)>0$$

The conditional PDF of $X$ given $A$ is defined as:

$$f_x(x|A)=\frac{d}{dx}{F}_X(x|A)$$

Computations

The expected value of the distribution is:

$${\mu }_X=E(X)={\int }_{-\infty }^{\infty }xf(x)dx$$

The variance is:

$${\sigma }_X^2=\text{Var}(X)={\int }_{-\infty }^{\infty }(x-{\mu }_X{)}^{2}f(x)dx={\int }_{-\infty }^{\infty }{x}^{2}f(x)dx-{\mu }_X^2$$

The $p$th percentile (where $p\in [0,100]$) is the point $x_p$ where:

$$F(x_p)={\int }_{-\infty }^{x_p}f(x)dx=\frac{p}{100}$$

The median is the 50th percentile, the point $x_m$ where:

$$F(x_m)={\int }_{-\infty }^{x_m}f(x)dx=0.5$$

When we compute the CDF, we should be cognisant of whether we’re able to do less work. For example, we might need to compute a big summation, but this could be simplified by computing 1 minus the complement instead.

Sub-pages

• Discrete distributions
  • Bernoulli distribution
  • Binomial distribution
  • Geometric distribution
  • Poisson distribution
  • Zipf distribution
• Continuous distributions
  • Exponential distribution
  • Gamma distribution
  • Beta distribution
  • Laplacian distribution
  • Cauchy distribution
  • Pareto distribution
  • Normal distribution (Gaussian distribution)