What problem does ElasticNet solve that neither Lasso nor Ridge can handle alone?

How to think about it

When a model predicts next-month customer spend, weekly_sessions and weekly_minutes might have a sample correlation of 0.98. The product team wants accurate predictions and a feature list that stays sensible when another month of data arrives. The practical answer is ElasticNet: it combines Lasso’s ability to create exact zeros with Ridge’s stabilising effect on correlated predictors. Lasso alone may choose one feature arbitrarily; Ridge alone generally keeps them all.

It is not that Lasso or Ridge cannot fit collinear data. Both can. The missing combination is sparse and stable selection when several predictors carry nearly the same information.

Why Lasso and Ridge behave differently

Collinearity means that two or more predictors move together so closely that the data cannot cleanly separate their individual contributions. If sessions and minutes rise together for nearly every customer, the model can estimate their combined signal more confidently than it can decide how much belongs to each column.

That matters because regression coefficients are not always independent discoveries. They are also an allocation of shared explanatory signal.

Lasso adds an L1 penalty, the sum of absolute coefficient values:

λ(|β₁| + |β₂| + ... + |βp|)

The absolute-value penalty has sharp corners. Those corners make the optimum land on an axis, which means one or more coefficients become exactly zero. That is why Lasso produces a sparse model, meaning a model with many exact zero coefficients.

The problem appears when predictors are highly correlated. If x₁ and x₂ are almost interchangeable, several allocations can produce nearly identical predictions. Lasso’s geometry favours an axis, so it may keep x₁ and discard x₂ in one sample, then do the reverse after a small change in the data. The choice can depend on tiny differences in the sample, column ordering, or the optimisation path.

Ridge adds an L2 penalty:

λ(β₁² + β₂² + ... + βp²)

Squaring makes large coefficients expensive and pulls coefficients toward zero, a process called shrinkage. But the squared penalty is smooth rather than cornered, so it does not normally make coefficients exactly zero. When two predictors share signal, Ridge is usually happy to give both of them some weight.

That gives the basic contrast:

Method	Correlated predictors	Exact zeros
Lasso	May choose one representative	Yes
Ridge	Usually shares weight across them	Usually no
ElasticNet	Encourages shared weight, while still selecting	Yes

The ElasticNet mechanism

ElasticNet minimises a prediction loss plus both penalties. A common form is:

minβ 1/(2n) ||y - Xβ||² + λ[r||β||₁ + (1-r)/2 ||β||₂²]

Here, X is the feature matrix, y is the target, β is the coefficient vector, n is the number of training rows, λ controls the overall regularisation strength, and r controls the mix between the two penalties.

The settings are:

• r = 1: pure Lasso
• r = 0: pure Ridge
• 0 < r < 1: ElasticNet

The L1 part, controlled by r, creates exact zeros. The L2 part discourages the model from assigning all of a shared signal to one arbitrary feature.

The grouping effect has a simple algebraic explanation. For two coefficients:

β₁² + β₂² = ((β₁ + β₂)² + (β₁ - β₂)²) / 2

If the prediction mainly depends on the sum β₁ + β₂, the L2 penalty still charges extra for making the difference β₁ - β₂ large. A split allocation is cheaper than an extreme allocation.

That is the mechanism an interviewer is probing for. ElasticNet does not merely average two models after training. The two penalties change the optimisation problem itself.

A numerical example

Suppose the predictors have been standardised, and for a toy problem x₁ and x₂ are identical. Let the target be:

y = 3x₁

Any coefficients satisfying β₁ + β₂ = 3 make the same prediction. Compare two possible allocations:

• One-feature allocation: β₁ = 3, β₂ = 0
• Shared allocation: β₁ = 1.5, β₂ = 1.5

For Lasso, both have the same L1 cost:

|3| + |0| = 3

and

|1.5| + |1.5| = 3

So Lasso has no penalty-based reason to prefer one allocation over the other. With perfectly duplicated columns, both can be optimal. With nearly duplicated columns, small noise usually breaks the tie.

For Ridge, the squared costs differ:

3² + 0² = 9

versus

1.5² + 1.5² = 4.5

Ridge prefers the shared allocation because it avoids one large coefficient.

ElasticNet retains the Lasso pressure for zeros but adds the Ridge preference for sharing. If the signal is strong enough, it will often keep both coefficients with similar magnitudes. If the regularisation is strong enough, it may set both to zero. For nearly, rather than perfectly, correlated features, equal coefficients are not guaranteed; the L2 term creates a tendency, not a rule.

How I would use it in practice

First, standardise numeric predictors. The penalty acts on coefficient values, so units matter. If one feature is measured in dollars and another in cents, their coefficients can differ by a factor of 100 even when they represent the same effect. Penalising those raw coefficients would make the result depend on the measuring stick.

Second, tune both regularisation parameters with cross-validation, meaning repeated training and validation splits used to estimate which settings generalise. Here is a leakage-safe scikit-learn pattern for a continuous target such as next-month spend:

from sklearn.linear_model import ElasticNet
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

search = GridSearchCV(
    estimator=Pipeline([
        ("scale", StandardScaler()),
        ("model", ElasticNet(max_iter=10000)),
    ]),
    param_grid={
        "model__alpha": [0.001, 0.01, 0.1, 1.0],
        "model__l1_ratio": [0.1, 0.5, 0.7, 0.9, 0.99],
    },
    scoring="neg_mean_squared_error",
    cv=5,
    n_jobs=-1,
)

search.fit(X_train, y_train)

best_model = search.best_estimator_.named_steps["model"]
print(search.best_params_)
print((best_model.coef_ != 0).sum())

In scikit-learn, the overall strength λ is called alpha, while the mixture r is called l1_ratio. That naming catches people: alpha is not the L1 proportion in this API.

The scaler belongs inside the searched pipeline. Otherwise, validation rows can influence the mean and standard deviation used during training. The leakage may be subtle, but the validation score is no longer a clean estimate. The example searches 4 × 5 parameter combinations across 5 folds, which means 100 fits. Larger logarithmic grids cost more, so compare that cost with the value of sparsity and stability. ElasticNetCV is a convenient specialised alternative; the important principle is that preprocessing and model selection must respect the fold boundaries.

For a binary churn target, use logistic regression with an elastic-net penalty rather than the ElasticNet regression estimator. The penalty idea is the same, but the prediction loss is different.

The senior nuance: it is not automatically the best model

ElasticNet is a compromise, not a trophy.

If predictors are weakly correlated and the business genuinely needs the smallest possible feature set, Lasso is simpler and may be easier to explain. If prediction is the only goal and many correlated measurements are useful, Ridge can achieve lower test error because it does not throw away redundant signal. I would fit both as baselines before claiming ElasticNet helped.

ElasticNet also does not know which features form meaningful domain groups. It sees columns and correlations. If ten one-hot columns represent one business concept and the requirement is to select or reject that concept as a unit, a group-aware method may be more appropriate.

The grouping effect is conditional. It is strongest for similarly scaled, strongly correlated predictors. Features with weaker correlation, different missingness patterns, opposing relationships, or a dominant L1 ratio may still be split. Saying “ElasticNet always selects correlated features together” is an interview red flag.

Finally, stable coefficients are not causal conclusions. If sessions and minutes are both retained, that means the penalised prediction problem found value in both. It does not prove that changing either one will increase spend. For that question, use an experiment or a causal design, not a regulariser.

Failure modes I would check first

The first symptom is unit-dependent selection. If changing minutes to hours changes which feature survives, the predictors were not scaled consistently. Put scaling in the training pipeline and inspect the transformed data.

The best parameter sits at the edge of the grid. If cross-validation chooses the smallest alpha, the search may not include weak enough regularisation. If it chooses the largest, it may need stronger values. If l1_ratio always lands at 0.1 or 0.99, expand the ratio grid or compare directly with Ridge and Lasso. A boundary result is information, not proof that the boundary is optimal.

A convergence warning appears, or coefficients change noticeably when max_iter increases. This can indicate poor scaling, a difficult correlated design, or an unsuitable regularisation range. Scaling and checking the grid should come before blindly increasing iterations.

Validation looks excellent but production error is poor. Look at the split. The same customer appearing in both training and validation, or random folds applied to time-ordered data, can make a regularised model look much better than it is.

What they’ll ask next

Does ElasticNet always select a correlated group together?
No. It encourages similar treatment through the L2 term. With perfectly duplicated, identically scaled predictors and a nonzero L2 component, symmetry tends to produce equal coefficients or zero coefficients together. Near-collinearity and noise can still produce unequal selections.

How do you choose alpha and l1_ratio?
Use cross-validation against the real objective, such as mean squared error, then check stability across resamples. alpha controls how strongly all coefficients are shrunk. l1_ratio controls the sparsity-versus-grouping trade-off. I would also compare against separately tuned Lasso and Ridge.

Why not fit Ridge and threshold small coefficients afterward?
Thresholding is a post-processing rule, not the same optimisation problem. The threshold is arbitrary, and removing one coefficient changes the value of the others without refitting. ElasticNet lets sparsity influence the coefficient allocation during training.

What if the target is classification?
Use a classifier with an elastic-net penalty, such as logistic regression configured for that penalty, rather than applying the regression estimator to class labels.

Say this in the interview

“ElasticNet is useful when correlated predictors make Lasso’s sparse selection unstable but Ridge’s solution too dense: the L1 term creates zeros, the L2 term shares signal across correlated features, and cross-validation chooses the balance.”