Bias-variance tradeoffΒΆ

The bias variance trade-off is maybe not an ideal name, it should maybe have better been called interpolation/extrapolation trade-off. Anyway, the motivation for the name is that that when adding more parameters / complexity, you have

  • Less systematic error (bias) in your model (supposedly, because it is more flexible, I would argue it depends on what you call error / bias)

  • More variance in the estimation of the model parameters (because it is more flexible)

Florian Hartig at Cross Validated. What is the difference between bias and residuals?