Expected Value and Variance

Overview

Expected value and variance are the two most important properties of a random variable, describing its center and spread.

Expected Value (Mean)

The expected value is the long-run average value of a random variable.

Discrete Random Variable

$$E(X) = \mu = \sum x_i \times P(X = x_i)$$

Continuous Random Variable

$$E(X) = \mu = \int x \times f(x) \, dx$$

Properties of Expected Value

$$E(c) = c \quad \text{(constant)}$$ $$E(cX) = c \times E(X) \quad \text{(scalar multiplication)}$$ $$E(X + Y) = E(X) + E(Y) \quad \text{(addition, always true)}$$ $$E(X - Y) = E(X) - E(Y) \quad \text{(subtraction)}$$ $$E(aX + b) = a \times E(X) + b \quad \text{(linear transformation)}$$

For independent $X$ and $Y$:

$$E(XY) = E(X) \times E(Y)$$

Variance

Variance measures the spread around the mean.

Definition

$$\text{Var}(X) = \sigma^2 = E[(X - \mu)^2]$$

Computational Formula

$$\text{Var}(X) = E(X^2) - [E(X)]^2$$

Discrete Random Variable

$$\text{Var}(X) = \sum (x_i - \mu)^2 \times P(X = x_i)$$

or equivalently:

$$\text{Var}(X) = \sum x_i^2 \times P(X = x_i) - \mu^2$$

Standard Deviation

$$\sigma = SD(X) = \sqrt{\text{Var}(X)}$$

Properties of Variance

$$\text{Var}(c) = 0 \quad \text{(constant has no variance)}$$ $$\text{Var}(cX) = c^2 \times \text{Var}(X) \quad \text{(scalar squared)}$$ $$\text{Var}(X + c) = \text{Var}(X) \quad \text{(shifting doesn't change spread)}$$ $$\text{Var}(aX + b) = a^2 \times \text{Var}(X) \quad \text{(linear transformation)}$$

For independent $X$ and $Y$:

$$\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y)$$ $$\text{Var}(X - Y) = \text{Var}(X) + \text{Var}(Y) \quad \text{(note: still addition!)}$$

Standardization

Converting $X$ to a standard form with mean 0 and SD 1:

$$Z = \frac{X - \mu}{\sigma}$$ $$E(Z) = 0, \quad \text{Var}(Z) = 1$$

Covariance and Correlation

Covariance

$$\text{Cov}(X, Y) = E[(X - \mu_X)(Y - \mu_Y)] = E(XY) - E(X)E(Y)$$

Correlation

$$\rho = \text{Corr}(X, Y) = \frac{\text{Cov}(X, Y)}{\sigma_X \times \sigma_Y}$$

Where $-1 \leq \rho \leq 1$

For Linear Combinations

$$E(aX + bY) = a \times E(X) + b \times E(Y)$$ $$\text{Var}(aX + bY) = a^2 \times \text{Var}(X) + b^2 \times \text{Var}(Y) + 2ab \times \text{Cov}(X,Y)$$

If $X$ and $Y$ are independent:

$$\text{Var}(aX + bY) = a^2 \times \text{Var}(X) + b^2 \times \text{Var}(Y)$$

Examples

Example 1: Expected Value

Die roll $X$:

$$E(X) = 1\left(\frac{1}{6}\right) + 2\left(\frac{1}{6}\right) + 3\left(\frac{1}{6}\right) + 4\left(\frac{1}{6}\right) + 5\left(\frac{1}{6}\right) + 6\left(\frac{1}{6}\right)$$ $$E(X) = \frac{21}{6} = 3.5$$

Example 2: Variance

For the die:

$$E(X^2) = 1^2\left(\frac{1}{6}\right) + 2^2\left(\frac{1}{6}\right) + 3^2\left(\frac{1}{6}\right) + 4^2\left(\frac{1}{6}\right) + 5^2\left(\frac{1}{6}\right) + 6^2\left(\frac{1}{6}\right)$$ $$E(X^2) = \frac{91}{6} = 15.17$$ $$\text{Var}(X) = 15.17 - (3.5)^2 = 15.17 - 12.25 = 2.92$$ $$\sigma = \sqrt{2.92} = 1.71$$

Example 3: Linear Transformation

If $E(X) = 50$ and $\text{Var}(X) = 16$, find $E(Y)$ and $\text{Var}(Y)$ where $Y = 3X + 10$:

$$E(Y) = 3 \times 50 + 10 = 160$$ $$\text{Var}(Y) = 3^2 \times 16 = 144$$ $$SD(Y) = 12$$

Example 4: Sum of Independent Variables

If $X$ has mean 100, SD 15 and $Y$ has mean 80, SD 10 (independent):

$$E(X + Y) = 100 + 80 = 180$$ $$\text{Var}(X + Y) = 225 + 100 = 325$$ $$SD(X + Y) = \sqrt{325} = 18.03$$