Causal inference for ML practitioners
Learn when a model can predict an outcome and when a design or estimator is needed to answer what an intervention will change.
What you'll learn
- Separate prediction from intervention using potential outcomes and the do operator.
- Diagnose confounding, Simpson's paradox, back-door paths, and collider bias.
- Choose and interpret propensity weighting, doubly-robust estimation, and difference-in-differences.
- Understand how T-learners and S-learners estimate conditional treatment effects.
- Recognise why unmeasured confounding cannot be tested away with more observational data.
Before you start
At 3:07 p.m., the growth dashboard says customers who received a discount spent an average of $42. Customers who did not receive one spent $31.
The marketing lead wants to send the discount to everyone.
There is a problem. The company did not choose recipients at random. It sent discounts to customers who abandoned carts, browsed expensive products, or looked likely to leave. Those customers were different before the discount arrived. The $11 gap mixes the effect of the discount with the reasons people were selected for it.
The real question is not, “What do customers who receive discounts spend?” It is, “What would happen to revenue if we assigned this customer the discount rather than not assigning it?”
That second question is causal inference.
Prediction is not intervention
A predictive model estimates patterns under the current policy: P(Y | X), the distribution of outcome Y given observed features X.
A causal question asks what happens after changing the assignment A: E[Y | do(A = 1)]. The do operator cuts the normal causes of A; we assign the offer instead of observing who happened to receive it. If assignment does not guarantee receipt, setting A to one changes the chance of receipt rather than forcing R to one.
Adding discount_received to a revenue model and scoring a customer twice—once with the feature set to one and once to zero—usually does not produce a causal estimate. If high-intent customers were more likely to receive discounts, the model can use the discount column as a proxy for intent. Changing the column does not remove that proxy; it asks the model to extrapolate to a feature combination rarely seen under the old policy.
A purchase-propensity model and an uplift model therefore answer different questions:
- Propensity asks who is likely to buy.
- Uplift asks whose behaviour the treatment will change.
Someone who would buy a $200 coat without a coupon has high propensity and nearly zero uplift.
Potential outcomes and the missing half of reality
For customer i, distinguish assignment from receipt:
A_i: whether the customer is assigned or offered the discount.R_i: whether the customer actually receives the discount, the binary actual-treatment variable.D_i: whether the customer redeems the discount after receiving it; redemption is distinct from receipt.Y_i(1): the outcome if assigned or offered the discount, including subsequent receipt and redemption behaviour.Y_i(0): the outcome if not assigned the discount.
The individual causal effect of assignment is Y_i(1) - Y_i(0). The assignment-level ATE is E[Y(1) - Y(0)]. This is not automatically the ATE of receipt, which requires separate receipt potential outcomes such as Y_i^R(1) and Y_i^R(0). It is also not the ATE of redemption D.
Each customer experiences only one world:
Y_i = A_iY_i(1) + (1-A_i)Y_i(0)
The other potential outcome is the counterfactual: what would have happened under the other action. This is the fundamental problem of causal inference.
An average effect can still be estimated if we can compare groups whose missing outcomes would have been comparable. That usually requires:
- Consistency. The intervention is well-defined, and the observed outcome matches its corresponding potential outcome. “A discount” is not one intervention if customers receive 5% or 40% off through different channels.
- No interference. One unit’s assignment does not change another unit’s outcome. This can fail in social networks, marketplaces, or contagious products.
- Conditional exchangeability. After accounting for baseline covariates
X, potential outcomes are independent of assignment:Y(1), Y(0) ⟂ A | X. - Positivity. Every relevant type of unit has a nonzero chance of each assignment. Data cannot identify what enterprise customers would do without a discount if every enterprise customer received one.
These are assumptions about the data-generating process, not library settings.
You may instead want the average treatment effect on the treated, or ATT: E[Y(1) - Y(0) | A = 1]. It asks how assigned customers would benefit, not how the whole customer base would benefit. ATE and ATT differ when effects vary.
Confounding, with arithmetic
A confounder is a pre-treatment variable that affects both assignment and outcome. Severity illustrates the problem. Drug A is better than Drug B in both severity groups, but doctors give A mostly to severe patients.
| Patient severity | Drug A patients | A recovered | A rate | Drug B patients | B recovered | B rate |
|---|---|---|---|---|---|---|
| Mild | 10 | 9 | 90% | 90 | 72 | 80% |
| Severe | 90 | 27 | 30% | 10 | 2 | 20% |
| Total | 100 | 36 | 36% | 100 | 74 | 74% |
Within both strata, A wins: 90% versus 80% for mild patients, and 30% versus 20% for severe patients. Overall, A recovers (9 + 27) / 100 = 36%, while B recovers (72 + 2) / 100 = 74%.
Drug A was used on 90 severe patients; B was used on only 10. The mixture changed, reversing the aggregate comparison. This is Simpson’s paradox: treatment groups contain different proportions of an outcome-predictive variable.
A model can predict the pattern accurately without making the comparison causal. Adjustment works only for the right variables, measured before treatment.
Randomisation is the gold standard
Random assignment breaks the connection between assignment A and measured or hidden baseline causes, at least in expectation. The difference in mean outcomes estimates the intention-to-treat effect:
ITT = E[Y | A = 1] - E[Y | A = 0]
Under randomisation and the experimental assumptions, this is the assignment-level ATE. It estimates the effect of offering the discount, not necessarily the effect of receiving or redeeming it. Noncompliance and leakage change the estimand, not the validity of ITT.
An A/B test randomly assigns eligible users, preserves a control arm, and compares a pre-declared outcome over a defined window. Randomisation does not make finite samples identical; it makes systematic baseline differences unlikely and quantifiable.
To estimate the effect of receipt, assignment can sometimes be an instrumental variable for R. The binary Wald estimate is:
τ_Wald = [E[Y | A = 1] - E[Y | A = 0]] / [E[R | A = 1] - E[R | A = 0]]
With relevance, exclusion—that assignment affects the outcome only through actual receipt R, including any downstream redemption—monotonicity, consistency, and no interference, this identifies a complier-average causal effect (LATE): the effect of receipt for customers whose receipt changes because of assignment. It does not identify the effect of redemption D. Do not divide an observed effect by redemption rate and call it the effect among redeemers. Redeemers can differ in motivation and purchase intent.
Choose the randomisation unit carefully. If treated users can influence controls, randomise by cluster, seller, region, or another unit that prevents interference.
When you cannot randomise
Propensity-score weighting
The propensity score is e(X) = P(A = 1 | X), the probability of assignment given pre-treatment covariates. Inverse-probability weighting gives assigned units weight 1/e(X) and unassigned units weight 1/(1-e(X)):
τ_IPW = (1/n) Σ [A_iY_i/e_i - (1-A_i)Y_i/(1-e_i)]
An assigned customer who had only a 10% chance of assignment counts roughly ten times; an unassigned customer who had a 90% chance also counts roughly ten times. The resulting pseudo-population should have less measured baseline imbalance.
Extreme propensities create enormous weights and unstable estimates. Inspect covariate balance before and after weighting, propensity overlap, maximum weights, and effective sample size. Propensity-model AUC is not the goal: high AUC can signal poor overlap.
Clipping probabilities is weight capping, not automatically trimming to a well-defined overlap population. If you trim or use overlap weighting, state the resulting estimand and rule.
Outcome regression and doubly robust estimation
Outcome regression models m_a(X) = E[Y | A = a, X], predicts each unit under assignment and control, and averages m_1(X) - m_0(X). It can extrapolate badly where one assignment is rare.
The augmented inverse-probability-weighted estimator is:
τ_DR = (1/n) Σ [m_1(X_i) - m_0(X_i) + A_i/e_i(Y_i - m_1(X_i)) - (1-A_i)/(1-e_i)(Y_i - m_0(X_i))]
Under the identification assumptions, it is consistent when either the propensity model or the outcome model is correctly specified. The residual terms correct outcome-model errors using assignment probabilities. If both models are wrong, hidden confounding remains, or propensities are extreme, the guarantee does not help. Cross-fitting—training nuisance models on one fold and predicting another—reduces overfitting.
If R is receipt recorded after assignment, do not substitute it for A by default. Use assignment-level ITT or a defensible IV strategy.
Difference-in-differences
Difference-in-differences (DiD) compares changes in a treated group with changes in a control group. If treated stores move from $100 to $130 and controls from $110 to $120:
(130 - 100) - (120 - 110) = 20
The identifying assumption is parallel trends: without treatment, treated stores would have changed like the controls. Groups need not have equal starting revenue, but their untreated trajectories must be comparable. Inspect several pre-treatment periods, and check anticipation, changing composition, and time-varying confounders. Staggered rollouts with heterogeneous effects require estimators designed for staggered adoption.
DAGs: draw the paths before adjusting
A DAG, or directed acyclic graph, represents assumed causal directions. It is not learned automatically from correlations.
If customer intent X affects both discount assignment A and buying Y, the graph contains X -> A, X -> Y, and A -> Y. The path through X is a back-door path because it enters A through an arrow pointing into assignment. A valid adjustment set blocks such paths without including descendants of A.
Including a variable in a regression, stratifying, matching, or weighting is adjustment. The graph—not predictive usefulness—should determine which variables belong.
Do not adjust for a mediator when estimating a total effect. If A -> R -> Y, controlling for post-treatment receipt R blocks part of the effect and can introduce further bias. A direct-effect analysis needs a separately defined estimand and stronger mediation assumptions.
A collider is a common effect: A -> C <- Y. Conditioning on admission, for example, can associate otherwise unrelated athletic training and academic performance because both affect admission. The same problem occurs with post-treatment selection fields and populations defined by both assignment and outcome.
Uplift and conditional treatment effects
A conditional average treatment effect is:
τ(x) = E[Y(1) - Y(0) | X = x]
CATE is an average effect for a subgroup or covariate profile, not the unknowable individual effect for one person. It helps decide whom to treat when effects vary. A high-propensity customer may buy without treatment and have little uplift; targeting should also account for treatment cost and harm.
A T-learner fits separate outcome models m_1(X) and m_0(X) on the two arms, then subtracts predictions. It is simple but can be noisy and extrapolate when either arm has poor overlap.
An S-learner fits one model m(X, A) using all observations and scores each unit twice, with A = 1 and A = 0. Shared data can improve stability, but regularisation may shrink treatment-by-feature interactions and produce nearly one effect for everyone.
Neither learner creates identification. Both still require credible exchangeability, positivity, consistency, and no interference. Evaluate a targeting policy by incremental policy value, V(π) = E[Y(π(X))], on randomized or otherwise valid evaluation data. Ordinary ROC AUC measures outcome ranking, not incremental benefit.
Inspect treatment effects, uncertainty, and policy value by subgroup. A policy that is profitable overall can concentrate missed benefits or harms in one group.
The limitation no estimator can hide
Unmeasured confounding is untestable from observational data alone.
Suppose a discount is assigned using private urgency. Urgency affects both assignment and purchase but is absent from the dataset. Weighting can balance every recorded column while leaving the important imbalance untouched.
More rows reduce sampling noise; they do not remove systematic bias. Sensitivity analysis can ask how strong an omitted confounder would need to be to erase the estimate. Negative controls, natural experiments, and instrumental variables can add leverage, but each requires its own assumptions.
State the estimand, target population, uncertainty interval, identifying assumptions, and what would change the conclusion.
What to remember
P(Y | X)predicts the observed policy;P(Y | do(A))asks what changes after intervention.- Each unit reveals only one potential outcome.
- Randomisation breaks assignment’s association with measured and hidden baseline causes.
- Weighting, doubly robust estimation, and DiD work only when their identifying assumptions fit the data.
- CATE models help choose whom to treat; they do not fix confounding or poor overlap.
Quick check
Practice this in an interview
All questionsWhen randomization is not feasible — due to ethical, operational, or technical constraints — quasi-experimental designs such as difference-in-differences, regression discontinuity, and synthetic control can recover causal estimates, but each requires strong and testable assumptions.
Maximum likelihood estimation finds the parameter values that make the observed data most probable under the assumed model. Intuitively, you ask: given this data, which world would have been most likely to generate it?
Open-ended ML problems require scoping before modelling: translate the vague ask into a measurable business objective, identify which user interaction has the highest improvement potential, formulate it as a concrete ML task with a defined label and evaluation metric, then propose the simplest viable model first. Jumping to model architecture before this scoping is the most common interview failure mode.
Bias can enter during problem framing, data collection, labeling, feature design, training, evaluation, and deployment. Mitigation includes better targets and sampling, label audits, proxy and leakage checks, weighted or constrained training, subgroup evaluation, thresholding, and production monitoring; deleting a protected attribute alone is not enough.