What is the difference between classification and regression, and how do you choose between them?

How to think about it

Short answer

Classification predicts a discrete category, such as fraud or not fraud. Regression predicts a numeric quantity, such as tomorrow’s demand or a house price. I choose between them by looking first at the target variable, the value the model must predict, and then at the business decision that prediction will support.

Why the distinction matters

The important difference is the model’s output space: the set of answers it is allowed to produce.

In classification, the target belongs to a finite set of classes. A binary classification problem has two classes, such as late and on-time. A multi-class problem has one class chosen from several possibilities, such as a support ticket labelled billing, technical, or account. A multi-label problem allows several labels at once, such as an image tagged both dog and outdoors.

A probabilistic classifier often estimates the chance of each class. For a fraud model, it might produce 0.82 for the fraud class. A threshold, a cutoff used to turn a score into an action, then converts that probability into a decision. The default cutoff might be 0.5, but it is not a law of nature. If missing a fraudulent transaction costs much more than reviewing a legitimate one, a threshold such as 0.1 may be more sensible.

That 0.82 does not make the problem regression. It is a continuous probability about a discrete event. The target is still fraud or not fraud.

In regression, the target is a numeric quantity. The model might predict 46.7 cartons, $312,400, or 18.2 minutes. Under mean-squared-error training, MSE being the average of squared prediction errors, a regression model tends to estimate the conditional mean: the average target value among cases with similar inputs. Under mean-absolute-error training, MAE being the average absolute error, it tends more toward the conditional median.

This difference affects the training objective and the evaluation. Classification commonly uses log-loss, accuracy, precision, recall, F1, or AUC. Regression commonly uses MAE, RMSE, or R². The metric should reflect the mistake the business actually cares about, not merely the metric that produces the most flattering number.

A concrete example: stocking milk

Imagine a supermarket deciding how much milk to order for tomorrow. The model receives the weekday, promotion calendar, weather forecast, recent sales, and current inventory.

There are two valid questions:

1. Will the store run out of milk tomorrow?
   The target is a class: stockout or no stockout. This is classification.

2. How many cartons will the store sell tomorrow?
   The target is a quantity, such as 46.7 expected cartons. This is regression, although a count-specific model may be more appropriate because actual demand is a non-negative integer.

The same inputs can support both models because the questions are different.

A classifier might be used like this:

p_stockout = stockout_model.predict_proba(X_today)[:, 1]
buy_one_extra = p_stockout >= 0.05

expected_demand = demand_model.predict(X_today)[0]

Suppose the classifier estimates an 0.82 probability of a stockout and the regression model predicts 46.7 cartons of demand.

The classification output is useful when the immediate action is a yes-or-no decision. In a simplified case, assume one extra carton costs $5, while allowing a stockout costs $100, and assume that extra carton prevents the stockout. Buying is worthwhile when:

probability of stockout × $100 is greater than $5.

That gives a threshold of 0.05. At a probability of 0.82, not buying has an expected stockout cost of $82, so buying the extra carton is clearly preferable.

The regression output is useful when the manager needs an order quantity. A forecast of 46.7 might lead to an order of 50 cartons after considering safety stock, delivery schedules, and waste. But the point forecast alone does not describe uncertainty. If running out is especially costly, the manager may need a 90th-percentile demand forecast rather than the average. A calibrated 90th-percentile forecast should be higher than demand on roughly 90 percent of comparable days.

How I choose

I start with the decision, then define the target that makes that decision possible.

Target or question	Usual starting point
One class from two choices	Binary classification
One class from several choices	Multi-class classification
Several labels can apply at once	Multi-label classification
A measured amount or score	Regression
An ordered label such as poor, fair, or good	Ordinal modelling
A non-negative event count	Poisson, negative-binomial, or another count model

The last two rows deserve care.

For poor, fair, and good, an ordinary multi-class classifier treats the labels as separate categories. It does not naturally understand that poor is closer to fair than to good. An ordinal model can use that ordering. Plain classification can still be acceptable when the business treats every wrong label equally, but it throws away useful structure when the distance between labels matters.

Counts are numeric, but they are not ordinary continuous measurements. A count model can enforce a non-negative expected value and represent the relationship between the mean and the variance. A standard regression model may still work well, especially with enough data, but it can produce negative predictions or ignore the fact that count variability often grows with the count itself. The fact that a target is stored as an integer does not automatically make it a classification problem.

Then I ask what kind of error matters.

For classification:

• Precision is the fraction of flagged cases that really belong to the positive class.
• Recall is the fraction of actual positive cases that the model finds.
• F1 balances precision and recall.
• AUC-ROC measures how well the model ranks positives above negatives across thresholds, but it does not choose the operating threshold or guarantee useful probabilities.
• Calibration means that predicted probabilities match observed frequencies. Among 100 cases predicted at 0.2, roughly 20 should be positive if the model is well calibrated.

For regression:

• MAE is the average absolute miss and stays in the target’s units.
• RMSE is the square root of average squared error, so a few large mistakes count more heavily.
• R² compares the model with a simple mean-prediction baseline. It is not a dollar measure and can be negative on unseen data.

If a retailer loses $100 from a stockout but only $5 from one extra carton, a fixed 0.5 classification threshold is hard to defend. The threshold should come from those costs, capacity constraints, review costs, and the reliability of the probabilities.

The nuance that earns the senior signal

Classification and regression describe the problem, not the algorithm family. Decision trees, random forests, gradient-boosting models, and neural networks all have classification and regression variants. The training loss and final output layer usually change, not the entire idea of the model.

Logistic regression is the classic naming trap. Despite the word “regression,” it is normally a classification model. It takes a linear score, passes it through a sigmoid function that maps the score into the range from zero to one, and interprets the result as a class probability. It is commonly trained with log-loss because that corresponds to modelling a binary outcome probabilistically.

Using mean-squared error on binary labels is not forbidden. A linear probability model can be useful in some settings. But an unbounded linear model can predict values below zero or above one, and MSE optimizes a different objective from logistic regression. The right response in an interview is not “MSE is impossible”; it is “I would choose the objective that matches the target and the decision, then validate calibration and operating performance.”

A continuous target can also be discretised. For example, predicting revenue above or below $1 million is classification, while predicting the revenue itself is regression. The classification version may be the right choice when the decision really is whether to approve a budget. But binning revenue discards information: $999,000 and $10,000 receive the same label even though they imply very different actions.

Conversely, a regression forecast can be thresholded to make a decision. Predicting demand and checking whether the prediction exceeds 40 cartons is reasonable, but it is not equivalent to training directly on the stockout boundary. Regression optimizes numeric accuracy; classification optimizes separation around the class decision.

A failure mode I would watch for

The first symptom of a bad classification setup is often impressive accuracy and terrible business results. Suppose only 10 of 1,000 store-days have stockouts. A model that always predicts no stockout achieves 99 percent accuracy, yet its recall is zero. The fix is to inspect the class distribution, use a sensible baseline, evaluate precision and recall, and choose a threshold using the cost of each error.

The first symptom of a bad regression setup may be a respectable average MAE while shelves still empty on peak days. Squared-error or absolute-error training often focuses on the centre of the demand distribution. If the expensive failures live in the upper tail, use a classification model for the stockout event or a quantile and probabilistic forecast that represents that tail directly.

What they’ll ask next

Can a classification model predict a continuous number?

It can output a continuous probability, but that probability is not the target quantity. Predicting the chance that a package is late is classification. Predicting how many minutes late it will be is regression. The first output supports a discrete action; the second estimates magnitude.

Why not use regression for every problem and apply a threshold afterward?

That can work, but the model was trained to minimise numeric error rather than to separate the classes at the decision boundary. If errors near the boundary matter most, direct classification may perform better. If the underlying quantity and its magnitude matter too, regression or a probabilistic model may preserve more useful information.

How do you handle an imbalanced classification problem?

I would not rely on accuracy. I would choose metrics and a threshold based on false-positive and false-negative costs, inspect precision, recall, and the precision-recall curve, and check probability calibration. I might also use class weighting or resampling, but those are tools, not substitutes for defining the right business objective.

Say this in the interview

“Classification predicts a category, regression predicts a quantity, and I choose between them from the target and the downstream cost of errors rather than from the algorithm’s name.”