Fairness & bias in ML
A model can be accurate overall and still systematically unfair across groups. Where bias comes from, the group-fairness metrics (demographic parity, equalized odds), and why you can't satisfy them all.
What you'll learn
- Where bias enters an ML system, and why 'accurate' isn't 'fair'
- Group-fairness metrics — demographic parity vs equalized odds
- The impossibility result, and mitigation with tools like Fairlearn
Before you start
A loan model with 92% accuracy sounds great — until you notice it approves one demographic group at twice the rate of another with the same qualifications. A model can be accurate overall and still systematically unfair across groups, and in 2026 — with the EU AI Act in force for high-risk systems and fair-lending laws already on the books — fairness is an engineering responsibility, not a footnote.
Where bias comes from
Models don’t invent bias; they inherit and amplify it. The usual sources:
- Biased labels / historical data. If past lending decisions were biased, a model trained to imitate them learns the bias. “Garbage in, bias out.”
- Sampling bias. Under-represented groups get worse predictions simply because the model saw fewer of them.
- Proxy features. Even after dropping a protected attribute, correlated features (zip code, name, device) let the model reconstruct it.
- Feedback loops. A biased model’s decisions shape future data, entrenching the bias over time.
Crucially, the bias often lives in the input scores and flows straight through a single decision threshold:
Measuring fairness — pick a definition
“Fair” isn’t one thing. The two most common group-fairness metrics ask different questions:
- Demographic parity — do groups get approved at the same rate? (Equal selection rate.) Ignores who’s actually qualified.
- Equalized odds — among the qualified, are groups approved at the same rate (equal true-positive rate), and likewise for the unqualified? Conditions on the ground truth.
Mitigation and accountability
You can intervene at three stages: pre-processing (reweight or rebalance the data), in-processing (add a fairness constraint to training), or post-processing (adjust decision thresholds per group, as the widget does). Microsoft’s open-source Fairlearn library implements all three plus a metrics dashboard. Pair the technical fix with model cards — short documents recording a model’s intended use, performance per group, and known limitations, which the EU AI Act increasingly expects as living evidence, not a one-time PDF.
Quick check
Quick check
Next
Fairness joins interpretability as the responsible-ML toolkit. To audit why a model decides, revisit SHAP and SHAP vs LIME.
Practice this in an interview
All questionsBias can enter through the data (historical, sampling, or labeling bias), the features (proxies for protected attributes), the objective (optimizing only for accuracy), and deployment (feedback loops). Mitigations are grouped into pre-processing (reweighting or resampling data), in-processing (adding fairness constraints during training), and post-processing (adjusting thresholds per group). Removing the protected attribute alone is insufficient because of proxy variables.
Demographic parity requires equal positive-prediction rates across groups, ignoring the true label; equalized odds requires equal true-positive and false-positive rates across groups, conditioning on the true label. In general you cannot satisfy both simultaneously (except in degenerate cases), because of impossibility results when base rates differ. Which metric to use depends on the harm you're trying to prevent.
Bias is error from oversimplifying assumptions (underfitting); variance is error from sensitivity to the training set (overfitting). Total error decomposes into bias squared, variance, and irreducible noise, and reducing one often increases the other. You diagnose by comparing training and validation error: high error on both means high bias, while a large gap (low train, high validation) means high variance.
The accuracy paradox occurs when a trivial model — one that always predicts the majority class — achieves high accuracy on an imbalanced dataset despite having zero predictive power for the minority class. A model that predicts 'not fraud' on every transaction achieves 99.9% accuracy if fraud is 0.1% of the data, but its recall for fraud is zero. Accuracy is only meaningful when classes are roughly balanced.