When should you use one-hot encoding versus label encoding for categorical features?

How to think about it

Use one-hot encoding for nominal categories, where the values are names with no meaningful order. Use ordinal encoding, often called label encoding in interviews, only when the categories have a genuine order and the model should use that order. For tree models, arbitrary integer codes can sometimes work, but native categorical handling or one-hot encoding is safer.

Why the choice matters

A model does not see “card”, “wallet”, or “bank transfer”. It sees numbers. The encoding decides what those numbers are allowed to mean.

A nominal feature is a categorical feature whose values have no natural ranking. Payment method is nominal: card is not “less than” wallet, and wallet is not halfway between wallet and bank transfer. Browser, city, colour, and product ID are also nominal.

An ordinal feature has a real order. Satisfaction might be low, medium, and high. Education level might have an order from secondary school to postgraduate study. The order carries information, although the gaps between levels may not be equal.

Label encoding replaces categories with integers. Suppose a training set maps:

card          -> 0
wallet        -> 1
bank_transfer -> 2

That mapping is harmless only if the model treats the integers as names. Many models do not. A linear model, for example, learns a formula such as logit(p) = b + w * code. If w is 0.8, moving from card to wallet increases the log-odds by 0.8, and moving from wallet to bank transfer increases it by the same amount.

There is no business reason for either statement. The numbers were invented by the preprocessing step.

One-hot encoding avoids that implication. It creates one binary indicator for each category:

payment method	card	wallet	bank transfer
card	1	0	0
wallet	0	1	0
bank transfer	0	0	1

A linear model can now learn a separate effect for each payment method. One possible fitted equation is logit(p) = b + 0.2 * card - 0.6 * wallet + 0.9 * bank_transfer. The coefficients describe categories independently rather than pretending that the categories lie on a ruler.

The same issue appears with distance. With label encoding, card and bank transfer are two units apart, while card and wallet are one unit apart. With one-hot vectors, every pair of different categories has Euclidean distance sqrt(2). That is usually the sensible geometry for a nominal feature.

This matters directly to KNN, K-means, distance-based SVMs, and neural networks. A neural network can sometimes learn around a poor encoding, but it has been given a misleading starting geometry. Making the network repair your feature representation is an expensive way to avoid choosing the right representation in the first place.

A concrete example

Imagine a churn model for a subscription service. Each customer has:

• payment_method: card, wallet, or bank transfer
• satisfaction: low, medium, or high
• monthly_spend: a numeric feature

payment_method is nominal. There is no meaningful ranking among the three methods, so one-hot encoding is appropriate.

satisfaction is ordinal. Low comes before medium, and medium comes before high, so ordinal encoding may be appropriate:

low    -> 0
medium -> 1
high   -> 2

A linear model will still make an additional assumption: the change from low to medium has the same numerical size as the change from medium to high. If that is not reasonable, one-hot encode satisfaction as well. An ordered feature does not automatically have equally spaced effects.

For example, perhaps churn falls sharply when satisfaction moves from low to medium, but barely changes from medium to high. One-hot encoding lets the model learn those two effects separately. Ordinal encoding with a single coefficient cannot express that shape without additional features.

In scikit-learn, a training pipeline could look like this:

from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder

preprocess = ColumnTransformer(
    transformers=[
        (
            "payment_method",
            OneHotEncoder(handle_unknown="ignore"),
            ["payment_method"],
        ),
        (
            "satisfaction",
            OrdinalEncoder(categories=[["low", "medium", "high"]]),
            ["satisfaction"],
        ),
    ]
)

model = Pipeline(
    steps=[
        ("preprocess", preprocess),
        ("classifier", LogisticRegression(max_iter=1000)),
    ]
)

model.fit(
    train[["payment_method", "satisfaction", "monthly_spend"]],
    y_train,
)

The explicit category list matters. Without it, the encoder may infer an order that is alphabetical rather than meaningful. “High”, “low”, and “medium” are not safely ordered by sorting their spelling.

For a feature column, OrdinalEncoder is the appropriate scikit-learn transformer. LabelEncoder is intended primarily for encoding a target vector, such as class labels in y. People often use “label encoding” as a generic phrase for integer encoding, but being precise here is a useful interview signal.

The decision rule

Situation	Usual choice	Reason
Nominal feature with a linear, distance-based, or neural model	One-hot	No artificial rank or distance
Truly ordered feature with a model that should use the rank	Ordinal	Preserves the order compactly
Binary feature	Zero and one	Only two values create a meaningful contrast
Low-cardinality nominal feature with a tree model	One-hot or native categorical support	Avoids arbitrary threshold order
High-cardinality nominal feature	Consider hashing, target encoding, embeddings, or native support	One column per category may be wasteful
User ID, transaction ID, or near-unique token	Usually neither directly	The feature is likely an identifier, not a useful category

Binary categories deserve a small clarification. Encoding has_discount as zero and one is fine. There is no problematic multi-level ordering when only two states exist; the number represents the contrast between the two states. The same idea applies to a binary nominal feature such as mobile versus desktop.

The tree-model nuance

A common interview answer says, “Label encoding is fine for tree models.” That answer is incomplete.

A conventional decision tree receiving payment codes 0, 1, and 2 splits using thresholds. It can make a split such as code <= 0.5, which isolates card, or code > 1.5, which isolates bank transfer. But it cannot represent every possible grouping in one split. If the useful grouping is card plus bank transfer versus wallet, the result depends on the arbitrary code order and may require multiple splits.

Change the mapping to card equals 2, wallet equals 0, and bank transfer equals 1, and the tree sees a different set of candidate partitions. The underlying customers have not changed. The model has.

One-hot encoding lets a tree split on one category indicator, although grouping several categories can still require several branches. Some modern tree libraries support categorical features natively. If that support is genuine and configured correctly, use it rather than manually forcing categories into integers.

So the practical answer is:

• Do not assume trees make arbitrary integers categorical.
• Use native categorical support where the library provides it.
• For a low-cardinality feature and a numeric-only tree API, one-hot encoding is the safer default.
• Integer encoding can be acceptable in a controlled case, especially for binary features, but it is not a universal rule.

The production traps

Fit the encoder on training data only. Put it inside a pipeline so that cross-validation fits the category vocabulary separately in each training fold. Fitting an encoder on the complete dataset before splitting allows test-only categories to influence the feature schema and can make validation less representative.

handle_unknown="ignore" prevents a prediction request from failing when production contains a category absent from training. The unfamiliar category is represented by zeros for that encoded feature. That is a useful safety mechanism, not a magic understanding of the new category. The model has no learned coefficient for it.

If the model has an intercept, an unknown category can even look like the omitted baseline category. If that distinction matters, map unknown values to an explicit unknown bucket and include that bucket during training.

Another trap is one-hot dimensionality. A city column with 20 cities produces 20 indicator columns. A product catalog with 500,000 products produces 500,000 possible columns. Sparse storage can make the first case cheap, but it does not make the second case automatically useful. A near-unique category often causes memorisation rather than generalisation.

Target encoding can be effective for high-cardinality features, but it replaces a category with a target statistic such as its mean conversion rate. It must be computed out of fold; otherwise the target for each training row leaks into its encoded feature. Hashing and learned embeddings are other options, depending on the model and the need to preserve category identity.

With one-hot encoding and an intercept, all indicators for a feature sum to one. That creates perfect redundancy. For an unregularized linear regression, you can drop one level with drop="first" and let the intercept represent that baseline. Regularized models can often tolerate all levels, and keeping all levels gives symmetric coefficients, so dropping one is not a law. It is a choice about identifiability and interpretation.

What they’ll ask next

“Can I use label encoding for a random forest?”

You can, but do not claim that the forest automatically understands categories. Numeric tree splits still depend on the assigned order. Use one-hot encoding for low-cardinality nominal features, or use a library’s native categorical feature support. The right choice also depends on cardinality and model implementation.

“What should I do when a new category appears in production?”

Design for it explicitly. Fit the encoder on training data, use an unknown-category policy such as handle_unknown="ignore" for one-hot features, and monitor the rate of unknown values. A sudden increase is a data-quality or distribution-shift signal, not merely a preprocessing detail.

“Should I always drop the first one-hot column?”

No. Drop one category when you need a non-redundant design matrix, especially for an unregularized linear model with an intercept. Keeping all columns is often fine with regularization, but the coefficients are then not uniquely interpretable. Whichever choice you make, document the reference category.

Say this in the interview: “I use one-hot encoding for nominal categories because integer codes create artificial order and distance; I use explicit ordinal encoding only when the rank is real, and for tree models I prefer native categorical handling or one-hot over arbitrary labels.”