Coefficient of Determination

Overview

The coefficient of determination, $R^2$, measures the proportion of variance in the dependent variable that is predictable from the independent variable(s).

Definition

$$R^2 = \frac{SSR}{SST} = 1 - \frac{SSE}{SST}$$

Where:

• $SSR$ = Sum of Squares Regression (explained)
• $SSE$ = Sum of Squares Error (unexplained)
• $SST$ = Sum of Squares Total

Relationship to Correlation

For simple linear regression:

$$R^2 = r^2$$

Where $r$ is the Pearson correlation coefficient.

Interpretation

$R^2$ Value	Interpretation
0%	Model explains none of the variance
25%	Model explains 25% of the variance
50%	Model explains 50% of the variance
75%	Model explains 75% of the variance
100%	Model explains all the variance

Example Statement

"$R^2 = 0.64$ means that 64% of the variation in $Y$ is explained by the linear relationship with $X$."

Properties

• Range: $0 \leq R^2 \leq 1$
• Non-negative (can be 0 but not negative)
• Higher $R^2$ = better fit
• Dimensionless (no units)

Adjusted $R^2$

For multiple regression, adjusted $R^2$ accounts for the number of predictors:

$$R^2_{\text{adj}} = 1 - \frac{(1 - R^2)(n - 1)}{n - k - 1}$$

Where:

• $n$ = sample size
• $k$ = number of predictors

Why Adjusted?

• Regular $R^2$ always increases with more predictors
• Adjusted $R^2$ penalizes for unnecessary variables
• Can decrease if new variable doesn't improve fit

Comparison

$R^2$	Adjusted $R^2$
Always increases with more variables	Can decrease
Compare models with same $n$	Compare models with different $k$
Simple regression	Multiple regression

Example Calculation

Given:

• $SST = 500$
• $SSE = 125$

$$R^2 = 1 - \frac{125}{500} = 1 - 0.25 = 0.75$$

The model explains 75% of the variance in $Y$.

Limitations

$R^2$ Can Be Misleading

1. Nonlinear relationships: High $R^2$ doesn't guarantee appropriateness
2. Outliers: Can artificially inflate or deflate $R^2$
3. Not about causation: High $R^2 \neq$ causal relationship
4. Sample-specific: May not generalize well

What $R^2$ Doesn't Tell You

• If the relationship is appropriate (linear vs nonlinear)
• If the predictors are correct
• If the model is good for prediction
• Whether assumptions are met

Good vs Bad $R^2$

"Good" $R^2$ depends on the field:

Field	Typical $R^2$
Physical sciences	0.90+
Biological sciences	0.70-0.90
Social sciences	0.50-0.70
Psychology/Marketing	0.30-0.50

Beyond $R^2$

Always examine:

• Residual plots
• Practical significance
• Prediction accuracy
• Model assumptions

A model with moderate $R^2$ may be more useful than one with high $R^2$ if it's simpler and more interpretable.