Independent probability

In probability theory, events are independent if one event occurring does not affect the probability of another event occurring. One basic axiom is that if events $A,B$ are independent, then this relation must hold:

$$P(A\cap B)=P(A)P(B)$$

Random variables

Multiple random variables

$X$ and $Y$ are independent random variables if any event $A\in X$ is independent of any event $B\in Y$, i.e.,:

$$P(X\text{ in }A,Y\text{ in }B)=P(X\text{ in }A)P(Y\text{ in }B)$$

If $X$ and $Y$ are independent, this implies that:

$$p_{X,Y}(x_j,y_k)=p_X(x_j)p_Y(y_k)$$

i.e., the joint PMF is equal to the product of the marginal PMFs.

The same holds true for the continuous case. For independent random variables $X,Y$, their joint CDF is equal to the product of its marginal CDFs.

$$F_{X,Y}(x,y)=F_X(x)F_Y(y)$$

They’re also independent if and only if their joint pdf is equal to the product of the marginal pdfs:

$$f_{X,Y}(x,y)=f_X(x)f_Y(y)$$

For a random variable $X$ with mean $\mu$, for $n$ independent, repeated measurements of $X$: $X_1,\cdots ,X_n$, we denote $X_j$ as independent, identically distributed (iid) random variables with the same pdf as $X$. We can use the sample mean of the sequence to estimate the value of $\mu$.