datarekha
Patterns June 10, 2026

AutoML raised the floor, not the ceiling

AutoGluon now tops the AutoML benchmark and beats rivals given a fraction of the time. That makes a strong baseline cheap — but the features and framing that actually win are exactly what AutoML can't automate.

6 min read · by datarekha · machine-learningautomlautogluontabular

There’s a reflex among engineers to dismiss AutoML as a toy. In 2026 that reflex is wrong. AutoML automates the whole tabular loop — preprocessing, model search, hyperparameter tuning, ensembling — and it does it well. The real skill isn’t avoiding it; it’s knowing exactly what it can and cannot do for you.

It’s a genuinely strong baseline now

The benchmark numbers are not marginal. AutoGluon’s official documentation states it is the state of the art in the AutoML Benchmark 2025, statistically significantly outperforming other AutoML systems, with a 5-minute budget beating rival frameworks given a full hour. Independent evaluation agrees: the peer-reviewed AutoML Benchmark (AMLB) in JMLR finds AutoGluon consistently has the highest average model-performance rank, while noting slower inference as a trade-off. Its trick is refreshingly simple — the AutoGluon-Tabular paper skips expensive pipeline/hyperparameter search in favor of ensembling many models via multi-layer stacking. For resource-constrained settings, Microsoft’s FLAML tunes frameworks like XGBoost and LightGBM efficiently with minimal overhead, and independent 2025 studies in Scientific Reports evaluate these tools across classification settings.

So run AutoML first: get a strong number in an hour, and read its leaderboard to learn which model families fit your data.

What it can’t do is what actually wins

AutoML searches over models, not ideas. The groupby-aggregation or the debt/income ratio that cracks the problem has to come from you — it can’t invent a feature from a column that doesn’t exist. And it offers no protection from a badly-framed problem: feed it leakage and it will exploit it perfectly, reporting a fantastic, fake score. It also produces a black box you still need to check for interpretability, calibration, and fairness.

That’s the real lesson. AutoML raised the floor: a competitive baseline is now an hour of compute. But the ceiling — the framing, the features, the judgment that separate good ML from bad — is exactly the part it can’t reach.

Skip to content