Probability

Probability is a measure of how likely an event is to occur.
The probability of an event ( A ) is denoted as:

$P(A)$

Probability Calculations

Event	Formula
Complement of A	$P(A^c) = 1 - P(A)$
Joint Probability (A and B)	$P(A \cap B)$
Union Probability (A or B)	$P(A \cup B) = P(A) + P(B) - P(A \cap B)$
Conditional Probability	$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$

Example

A bag contains 10 balls: 6 red and 4 blue. Two balls are drawn randomly without replacement.

Let:

Event ( A ): First ball is red
Event ( B ): Both balls are red

1) Joint Probability

Probability first ball is red:

$P(A) = \frac{6}{10}$

Probability second ball is red given first is red:

$P(B \mid A) = \frac{5}{9}$

Joint probability:

$P(A \cap B) = P(A) \cdot P(B \mid A) = \frac{6}{10} \cdot \frac{5}{9} = \frac{1}{3}$

2) Conditional Probability

$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$

Since $( B \subseteq A )$, we have:

$P(B) = \frac{6}{10} \cdot \frac{5}{9} = \frac{1}{3}$

Thus:

$P(A \mid B) = 1$

Probability Distributions

A probability distribution describes how probabilities are assigned to the values of a random variable.

Uniform Distribution

A uniform distribution assumes all values in an interval are equally likely.

$X \sim U(a, b)$

Probability density function (PDF):

$f(x) = \begin{cases} \frac{1}{b - a}, & a \le x \le b \\ 0, & \text{otherwise} \end{cases}$

Bernoulli Distribution

A Bernoulli distribution models a binary outcome (success/failure).

$X = \begin{cases} 1, & \text{with probability } p \\ 0, & \text{with probability } 1 - p \end{cases}$

Probability mass function (PMF):

$P(X = x) = p^x (1 - p)^{1 - x}, \quad x \in \{0,1\}$

Binomial Distribution

The binomial distribution models the number of successes in ( n ) independent Bernoulli trials.

$X \sim \text{Binomial}(n, p)$

Probability mass function:

$P(X = k) = \binom{n}{k} p^k (1 - p)^{n-k}$

Where:

( n ): number of trials
( k ): number of successes
( p ): probability of success

Binomial Distribution

Normal Distribution

The normal (Gaussian) distribution is a continuous probability distribution.

$X \sim \mathcal{N}(\mu, \sigma^2)$

Probability density function:

$f(x) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left(-\frac{(x - \mu)^2}{2\sigma^2}\right)$

( \mu ): mean
( \sigma^2 ): variance

Standard normal distribution:

$\mathcal{N}(0,1)$

Normal Distribution

Central Limit Theorem

$\bar{X} \xrightarrow{d} \mathcal{N}(\mu, \sigma^2 / n)$

Bayes’ Theorem

$P(B) = \sum_{i=1}^{n} P(B \mid A_i) P(A_i)$

Bayes’ Formula

$P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$

Where:

Posterior: ( P(A $\mid$ B) )
Likelihood: ( P(B $\mid$ A) )
Prior: ( P(A) )
Evidence: ( P(B) )

Likelihood Function

$L(\theta \mid x) = P(x \mid \theta)$

For independent samples:

$L(\theta) = \prod_{i=1}^{n} P(x_i \mid \theta)$

Log-likelihood:

$\ell(\theta) = \sum_{i=1}^{n} \log P(x_i \mid \theta)$

Maximum Likelihood Estimation

$\hat{\theta} = \arg\max_{\theta} L(\theta)$