Probability Fundamentals

Overview

Probability is the mathematical framework for quantifying uncertainty. It forms the foundation for statistical inference.

Basic Terminology

Term	Definition
Experiment	A process with uncertain outcomes
Sample Space ($S$)	Set of all possible outcomes
Event	A subset of the sample space
Outcome	A single result of an experiment

Probability Axioms

For any event $A$ in sample space $S$:

1. Non-negativity: $P(A) \geq 0$
2. Normalization: $P(S) = 1$
3. Additivity: If $A$ and $B$ are mutually exclusive:

$$P(A \cup B) = P(A) + P(B)$$

Probability Rules

Classical Probability

When outcomes are equally likely:

$$P(A) = \frac{\text{Number of favorable outcomes}}{\text{Total number of outcomes}} = \frac{n(A)}{n(S)}$$

Complement Rule

$$P(A') = 1 - P(A)$$

Where $A'$ (or $\bar{A}$) is "not A".

Addition Rule

For any events $A$ and $B$:

$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$

For mutually exclusive events:

$$P(A \cup B) = P(A) + P(B)$$

Multiplication Rule

For independent events:

$$P(A \cap B) = P(A) \times P(B)$$

For dependent events:

$$P(A \cap B) = P(A) \times P(B|A)$$

Set Operations in Probability

Operation	Symbol	Meaning
Union	$A \cup B$	A or B (or both)
Intersection	$A \cap B$	A and B
Complement	$A'$ or $\bar{A}$	Not A
Difference	$A - B$	A but not B

Types of Events

Type	Definition	Property
Mutually Exclusive	Cannot occur together	$P(A \cap B) = 0$
Independent	One doesn't affect other	$P(A \cap B) = P(A) \cdot P(B)$
Exhaustive	Cover entire sample space	$P(A \cup B \cup \ldots) = 1$
Complementary	Mutually exclusive + exhaustive	$P(A) + P(A') = 1$

Common Probability Values

Event	Probability
Impossible	0
Certain	1
Fair coin heads	0.5
Fair die showing 6	1/6 ≈ 0.167
At least one head in 2 flips	0.75

Examples

Example 1: Basic Probability

Single die roll:

$$S = \{1, 2, 3, 4, 5, 6\}$$ $$P(\text{even}) = P(\{2, 4, 6\}) = \frac{3}{6} = 0.5$$

Example 2: Complement Rule

$$P(\text{at least one 6 in 3 rolls}) = 1 - P(\text{no 6s})$$ $$= 1 - \left(\frac{5}{6}\right)^3 = 1 - 0.579 = 0.421$$

Example 3: Addition Rule

In a class: 60% study math, 50% study physics, 30% study both.

$$P(\text{math or physics}) = 0.60 + 0.50 - 0.30 = 0.80$$

Example 4: Multiplication (Independent)

Two coin flips:

$$P(\text{both heads}) = P(H) \times P(H) = 0.5 \times 0.5 = 0.25$$

Example 5: Cards

$$P(\text{ace}) = \frac{4}{52} = \frac{1}{13}$$ $$P(\text{heart}) = \frac{13}{52} = \frac{1}{4}$$ $$P(\text{ace and heart}) = \frac{1}{52}$$ $$P(\text{ace or heart}) = \frac{4}{52} + \frac{13}{52} - \frac{1}{52} = \frac{16}{52} = \frac{4}{13}$$

Probability Distribution Table

For discrete random variable $X$:

Requirement	Description
All $P(X = x) \geq 0$	Probabilities are non-negative
$\sum P(X = x) = 1$	Total probability equals 1