Explain the bias-variance tradeoff and how it relates to overfitting.

For Data Scientist ML Engineer research-engineer

The short answer

Bias is error from overly simple assumptions (underfitting) and variance is error from sensitivity to training-data noise (overfitting); reducing one often increases the other. An overfit model has low bias but high variance, so techniques like regularization, more data, and simpler models trade a little bias for a large reduction in variance.

How to think about it

Bias is error from overly simple assumptions (underfitting) and variance is error from sensitivity to training-data noise (overfitting); reducing one often increases the other. An overfit model has low bias but high variance, so techniques like regularization, more data, and simpler models trade a little bias for a large reduction in variance.

Learn it properly Overfitting & bias–variance

Keep practising

What is the bias–variance tradeoff? Explain the bias-variance tradeoff and how you'd diagnose which one you have. How do L1 and L2 regularization affect bias and variance, and when would you pick one over the other? What are overfitting and underfitting, and how do you fix each? How do you select the regularization strength λ, and what does it mean to set it too high or too low?

All Deep Learning questions

Explore further

The Bias-Variance Trade-off Bias–variance & learning curves Fairness & bias in ML

Bias-Variance Tradeoff Overfitting Underfitting Sampling Bias