What are the core assumptions of linear regression, and what breaks when each is violated?
OLS linear regression rests on five assumptions: linearity, independence of errors, homoscedasticity, normality of residuals, and no perfect multicollinearity. Violating any one of them degrades coefficient estimates, standard errors, or the validity of hypothesis tests.
How to think about it
Linear regression’s Ordinary Least Squares estimator is BLUE (Best Linear Unbiased Estimator) only when its five Gauss-Markov assumptions hold.
The five assumptions and their failure modes:
-
Linearity — the relationship between predictors and response is linear. Violation: systematic curvature in residual plots; fix with feature transforms or polynomial terms.
-
Independence of errors — residuals are not autocorrelated. Violation (common in time-series): Durbin-Watson statistic departs from 2; OLS standard errors are understated.
-
Homoscedasticity — error variance is constant across all fitted values. Violation (heteroscedasticity): Breusch-Pagan test flags it; inference breaks even though coefficient estimates remain unbiased.
-
Normality of residuals — residuals follow a normal distribution. Required for exact finite-sample inference; with large n the Central Limit Theorem rescues hypothesis tests.
-
No perfect multicollinearity — no predictor is an exact linear combination of others. Near-multicollinearity inflates variance of estimates (check VIF).
Quick diagnostics in Python:
import statsmodels.api as sm
model = sm.OLS(y, sm.add_constant(X)).fit()
print(model.summary()) # Durbin-Watson, F-stat, R²
sm.graphics.plot_regress_exog(model, "x1") # residual plotsViolation of linearity is structural and cannot be patched by standard errors alone — transform your features or switch models.