Correlation

Overview

Correlation measures the strength and direction of the linear relationship between two quantitative variables.

Pearson Correlation Coefficient ($r$)

Formula

$$r = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum (x_i - \bar{x})^2 \times \sum (y_i - \bar{y})^2}}$$

Or equivalently:

$$r = \frac{n\sum x_i y_i - (\sum x_i)(\sum y_i)}{\sqrt{[n\sum x_i^2 - (\sum x_i)^2][n\sum y_i^2 - (\sum y_i)^2]}}$$

Properties

• Range: $-1 \leq r \leq 1$
• $r = 1$: Perfect positive linear relationship
• $r = -1$: Perfect negative linear relationship
• $r = 0$: No linear relationship

Interpretation

$\lvert r \rvert$	Strength
0.00 - 0.19	Very weak
0.20 - 0.39	Weak
0.40 - 0.59	Moderate
0.60 - 0.79	Strong
0.80 - 1.00	Very strong

Direction

$r$ Sign	Direction	Interpretation
$r > 0$	Positive	As $X$ increases, $Y$ tends to increase
$r < 0$	Negative	As $X$ increases, $Y$ tends to decrease

Spearman's Rank Correlation ($\rho$)

For ordinal data or non-linear relationships:

$$\rho = 1 - \frac{6\sum d_i^2}{n(n^2 - 1)}$$

Where $d_i$ = difference between ranks of corresponding values.

Assumptions (Pearson)

1. Continuous data
2. Linear relationship
3. Bivariate normality (for inference)
4. No significant outliers

Hypothesis Testing

Hypotheses

• $H_0$: $\rho = 0$ (no linear correlation)
• $H_1$: $\rho \neq 0$ (or one-tailed alternative)

Test Statistic

$$t = \frac{r\sqrt{n-2}}{\sqrt{1-r^2}}$$

With $df = n - 2$

Examples

Example 1: Calculating $r$

$x$	$y$
2	4
3	5
5	7
7	9
8	10

$$\sum x = 25, \quad \sum y = 35, \quad \sum xy = 193$$ $$\sum x^2 = 151, \quad \sum y^2 = 271, \quad n = 5$$ $$r = \frac{5(193) - (25)(35)}{\sqrt{[5(151) - 625][5(271) - 1225]}}$$ $$= \frac{965 - 875}{\sqrt{(130)(130)}} = \frac{90}{130} = 0.692$$

Strong positive correlation.

Example 2: Testing Significance

$r = 0.65$, $n = 20$, $\alpha = 0.05$

$$t = \frac{0.65\sqrt{20-2}}{\sqrt{1 - 0.65^2}} = \frac{0.65 \times 4.243}{\sqrt{0.5775}} = \frac{2.76}{0.76} = 3.63$$ $$df = 18, \quad t_{\text{crit}} = 2.101$$ $$3.63 > 2.101 \Rightarrow \text{Reject } H_0$$

Significant correlation exists.

Important Cautions

Correlation ≠ Causation

Just because $X$ and $Y$ are correlated does NOT mean:

• $X$ causes $Y$
• $Y$ causes $X$
• There's any causal connection

A third variable may explain both (confounding).

Restricted Range

Limiting the range of $X$ or $Y$ artificially reduces $r$.

Outliers

Single outliers can dramatically change $r$.

Nonlinear Relationships

$r$ only measures linear relationships. A perfect curve may have $r \approx 0$.

Coefficient of Determination

$$r^2 = \text{coefficient of determination}$$

Interpretation: The proportion of variance in $Y$ explained by $X$.

Example: $r = 0.8 \Rightarrow r^2 = 0.64 \Rightarrow$ 64% of variance in $Y$ is explained by $X$.