Analysis of Variance (ANOVA)

Overview

ANOVA tests whether the means of three or more groups are significantly different. It compares variance between groups to variance within groups.

One-Way ANOVA

Hypotheses

• $H_0$: $\mu_1 = \mu_2 = \mu_3 = \cdots = \mu_k$ (all means equal)
• $H_1$: At least one mean is different

The F-Statistic

$$F = \frac{MSB}{MSW} = \frac{\text{Between-group variance}}{\text{Within-group variance}}$$

Large $F$ → Evidence of differences between groups

ANOVA Table

Source	SS	df	MS	F
Between	SSB	$k - 1$	$MSB = \frac{SSB}{k-1}$	$\frac{MSB}{MSW}$
Within	SSW	$N - k$	$MSW = \frac{SSW}{N-k}$
Total	SST	$N - 1$

Where:

• $k$ = number of groups
• $N$ = total number of observations

Sum of Squares Formulas

Total Sum of Squares

$$SST = \sum (x_{ij} - \bar{x})^2$$

Between-Group Sum of Squares

$$SSB = \sum n_i (\bar{x}_i - \bar{x})^2$$

Within-Group Sum of Squares

$$SSW = \sum \sum (x_{ij} - \bar{x}_i)^2$$

Relationship

$$SST = SSB + SSW$$

Assumptions

1. Independence of observations
2. Normal distributions in each group
3. Equal variances (homogeneity of variance)

Example

Three teaching methods, student scores:

Method A	Method B	Method C
85, 90, 78	92, 88, 95	75, 70, 80
$n_1 = 3$	$n_2 = 3$	$n_3 = 3$
$\bar{x}_1 = 84.33$	$\bar{x}_2 = 91.67$	$\bar{x}_3 = 75$

Grand mean: $\bar{x} = \frac{85+90+78+92+88+95+75+70+80}{9} = 83.67$

$$SSB = 3(84.33-83.67)^2 + 3(91.67-83.67)^2 + 3(75-83.67)^2$$ $$= 3(0.44) + 3(64) + 3(75.11) = 1.32 + 192 + 225.33 = 418.65$$ $$SSW = (85-84.33)^2 + (90-84.33)^2 + (78-84.33)^2 + \cdots = 170.67$$ $$MSB = \frac{418.65}{2} = 209.33$$ $$MSW = \frac{170.67}{6} = 28.44$$ $$F = \frac{209.33}{28.44} = 7.36$$

Critical $F_{0.05, 2, 6} = 5.14$

$7.36 > 5.14 \Rightarrow$ Reject $H_0$

At least one teaching method differs significantly.

Post-Hoc Tests

If ANOVA rejects $H_0$, use post-hoc tests to identify which pairs differ:

Test	Use Case
Tukey's HSD	All pairwise comparisons
Bonferroni	Few specific comparisons
Scheffé	Complex comparisons
Fisher's LSD	Liberal, more Type I error

Effect Size: Eta-Squared

$$\eta^2 = \frac{SSB}{SST}$$

$\eta^2$	Interpretation
0.01	Small
0.06	Medium
0.14	Large

Two-Way ANOVA

For two factors (A and B):

$$SST = SS_A + SS_B + SS_{AB} + SSW$$

Tests:

• Main effect of Factor A
• Main effect of Factor B
• Interaction effect ($A \times B$)

Assumptions Check

• Normality: Shapiro-Wilk test, Q-Q plots
• Equal variances: Levene's test, Bartlett's test

When to Use

Situation	Test
2 groups	t-test
3+ groups	One-way ANOVA
2 factors	Two-way ANOVA
Repeated measures	Repeated measures ANOVA