Explain the relationship between the sigmoid function, odds, and log-odds in logistic regression.
Logistic regression models log-odds as a linear function of the features. Exponentiating the coefficients gives odds ratios, and applying the sigmoid to the linear score converts it to a probability. These three representations are equivalent reformulations of the same model.
How to think about it
Three equivalent views of the same model:
Probability — the sigmoid maps a real-valued score to (0,1):
p = σ(Xβ) = 1 / (1 + e^(-Xβ))
Odds — the ratio of success probability to failure probability:
odds = p / (1 - p)
Substituting the sigmoid: odds = e^(Xβ), so odds are exponential in the linear combination.
Log-odds (logit) — the natural log of the odds:
log(p / (1 - p)) = Xβ
This is exactly the linear model. Logistic regression is a linear model in log-odds space.
Interpreting coefficients:
A one-unit increase in feature xⱼ multiplies the odds by e^(βⱼ). If βⱼ = 0.5, the odds increase by a factor of e^0.5 ≈ 1.65 — a 65% increase in odds (not a 65% increase in probability).
import numpy as np
beta = model.coef_[0]
odds_ratios = np.exp(beta) # one-unit increase in xj multiplies odds by this