ECE302 — Probability and Applications

Introduction to basic concepts in probability theory. Pre-requisite to many upper-year machine learning courses.

We used the Probability, Statistics, and Random Processes for Electrical Engineering textbook, by Alberto Leon-Garcia.

Concepts covered

• Basic probability theory
  • Mutual exclusivity
  • Events
  • Basic set operations ($\wedge$, $\vee$, $\backslash$) and De Morgan’s laws
  • Finite and infinite sets (sample spaces)
  • Tree diagrams
  • Conditional probability
    • Bayes’ theorem
  • Classification metrics (accuracy)
• Combinatorics
  • Rule of products, rule of sums
  • Sampling {with, without} replacement, {with, without} ordering
  • Permutations
  • Combinations
• Markov chain
• Random variable
  • Functions of a random variable
  • Multiple random variables
    • Joint PMFs, PDFs, CDFs
    • Joint moments
    • Correlation
      • Pearson correlation coefficient
    • Covariance
• Probability distribution
  • Probability mass function (PMF)
  • Probability density function (PDF)
  • Cumulative distribution function (CDF)
  • Discrete probability distributions
    • Bernoulli distribution
      • Bernoulli trials
    • Binomial distribution
      • Binomial probability law
      • Binomial theorem
    • Geometric distribution
    • Poisson distribution
    • Zipf distribution
  • Continuous probability distributions
    • Exponential distribution
    • Gamma distribution
    • Beta distribution
    • Cauchy distribution
    • Pareto distribution
    • Normal distribution
• Metrics
  • Expected value (mean)
  • Variance
  • Standard deviation
• Transform methods
  • Characteristic function
    • Continuous-time Fourier transform
    • Discrete-time Fourier transform
  • Probability generating function
    • Z-transform
  • Laplace transform
• Shannon entropy (and uncertainty)
  • Huffman compression