Ordinary least squares, OLS

OLS is at the core of econometrics curriculum, it is easily derived and highly practical to familiarise a learner with regression possibilites and limitations.

The usual way to teach OLS is to present assumptions and show how to deal with their violations as indicated below in a review chart from Kennedy’s textbook.



\(Y = \beta X + \epsilon\), \(\epsilon\) is iid, normal with finite variance.

Common steps:

  1. specify model: select explanatory variables, transform them if needed

  2. estimate coefficients

  3. elaborate on model quality (the hardest part)

  4. go to 1 if needed

  5. know what model does not show (next hardeer part)

What may go wrong:

  • residuals are not random

  • variables are cointegrated

  • multicollinearity in regressors

  • residuals depend on x (heteroscedasticity)

  • inference is not causality

  • wrong signs, insignificant coefficients

  • variable normalisation was not described


  • why sum of squares as a loss function?

  • connections to bayesian estimation

  • is R2 useful or dangerous?


Baby dragon special: