Hypothesis Testing Basics

Overview

Hypothesis testing is a formal procedure for using sample data to evaluate claims about population parameters.

The Process

1. State hypotheses ($H_0$ and $H_1$)
2. Choose significance level ($\alpha$)
3. Collect data and calculate test statistic
4. Find p-value or critical value
5. Make decision and state conclusion

Hypotheses

Null Hypothesis ($H_0$)

The default assumption, typically:

• No effect
• No difference
• Status quo

Alternative Hypothesis ($H_1$ or $H_a$)

What we're trying to find evidence for:

• There is an effect
• There is a difference
• Something has changed

Types of Tests

Test Type	$H_0$	$H_1$	Rejection Region
Two-tailed	$\mu = \mu_0$	$\mu \neq \mu_0$	Both tails
Left-tailed	$\mu \geq \mu_0$	$\mu < \mu_0$	Left tail
Right-tailed	$\mu \leq \mu_0$	$\mu > \mu_0$	Right tail

Significance Level ($\alpha$)

The probability of rejecting $H_0$ when it's actually true (Type I error rate).

Common values: $\alpha = 0.05$, $0.01$, $0.10$

Test Statistic

For Means ($\sigma$ known or large $n$)

$$Z = \frac{\bar{x} - \mu_0}{\sigma/\sqrt{n}}$$

For Means ($\sigma$ unknown, small $n$)

$$t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}$$

For Proportions

$$Z = \frac{\hat{p} - p_0}{\sqrt{p_0(1-p_0)/n}}$$

P-Value

The probability of observing a test statistic as extreme or more extreme than the one calculated, assuming $H_0$ is true.

Decision Rule

$$\text{If p-value} \leq \alpha \Rightarrow \text{Reject } H_0$$ $$\text{If p-value} > \alpha \Rightarrow \text{Fail to reject } H_0$$

Critical Value Approach

Compare test statistic to critical value(s):

$$\text{If } \lvert \text{test stat} \rvert > \text{critical value} \Rightarrow \text{Reject } H_0$$

Decision Outcomes

	$H_0$ True	$H_0$ False
Reject $H_0$	Type I Error ($\alpha$)	Correct
Fail to Reject	Correct	Type II Error ($\beta$)

Steps Illustrated

         State H₀, H₁
              ↓
         Choose α
              ↓
         Collect data
              ↓
         Calculate test statistic
              ↓
         Find p-value
              ↓
    ┌────────┴────────┐
    ↓                 ↓
p ≤ α              p > α
Reject H₀      Fail to reject H₀

Examples

Example 1: Two-Tailed Test

Claim: $\mu = 100$. Sample: $n = 36$, $\bar{x} = 103$, $\sigma = 12$, $\alpha = 0.05$

$$H_0: \mu = 100$$ $$H_1: \mu \neq 100$$ $$Z = \frac{103 - 100}{12/\sqrt{36}} = \frac{3}{2} = 1.5$$ $$\text{p-value} = 2 \times P(Z > 1.5) = 2 \times 0.0668 = 0.1336$$ $$0.1336 > 0.05 \Rightarrow \text{Fail to reject } H_0$$

Example 2: One-Tailed Test

Claim: $\mu > 50$. Sample: $n = 25$, $\bar{x} = 53$, $s = 8$, $\alpha = 0.05$

$$H_0: \mu \leq 50, \quad H_1: \mu > 50$$ $$t = \frac{53 - 50}{8/\sqrt{25}} = \frac{3}{1.6} = 1.875$$ $$df = 24, \quad t_{\text{crit}} = 1.711$$ $$1.875 > 1.711 \Rightarrow \text{Reject } H_0$$

Example 3: Proportion Test

Claim: $p > 0.5$. Sample: $n = 200$, $\hat{p} = 0.56$, $\alpha = 0.05$

$$H_0: p \leq 0.5, \quad H_1: p > 0.5$$ $$Z = \frac{0.56 - 0.5}{\sqrt{0.5 \times 0.5/200}} = \frac{0.06}{0.0354} = 1.69$$ $$\text{p-value} = P(Z > 1.69) = 0.0455$$ $$0.0455 < 0.05 \Rightarrow \text{Reject } H_0$$

Key Points

• We never "prove" $H_0$ or "accept" $H_0$
• Failing to reject ≠ proving true
• Statistical significance ≠ practical significance
• Choose $\alpha$ before analyzing data