Question 1

What is the accuracy paradox and how does it expose the failure of accuracy as a metric?

Accepted Answer

The accuracy paradox occurs when a trivial model — one that always predicts the majority class — achieves high accuracy on an imbalanced dataset despite having zero predictive power for the minority class. A model that predicts 'not fraud' on every transaction achieves 99.9% accuracy if fraud is 0.1% of the data, but its recall for fraud is zero. Accuracy is only meaningful when classes are roughly balanced.

Question 2

What is the difference between classification and regression, and how do you choose between them?

Accepted Answer

Classification predicts a discrete class label; regression predicts a continuous numeric value. The choice is determined by the nature of the target variable, not by the algorithm family — many algorithms (e.g., decision trees, neural nets) handle both.

Question 3

What is a confusion matrix and what four quantities does it report?

Accepted Answer

A confusion matrix tallies predictions against ground truth in a 2x2 table: true positives, true negatives, false positives, and false negatives. From those four cells every classification metric — accuracy, precision, recall, F1, specificity — can be derived. It exposes *which kind* of error a model makes, not just how often it errs.

Question 4

How do you extract useful features from datetime columns for a machine learning model?

Accepted Answer

Raw timestamps are meaningless to most models. Useful features extracted from a datetime column include calendar components (hour, day of week, month, quarter, year), cyclical encodings of periodic components (sin/cos of hour or day-of-week), lag and rolling-window aggregates, time-since-event features, and business-calendar flags like is_weekend or is_holiday.

Question 5

What is the difference between Gini impurity and entropy as splitting criteria in decision trees?

Accepted Answer

Both measure node impurity but differ in computation and sensitivity. Gini is faster to compute and slightly favors larger partitions, while entropy (information gain) is more sensitive to class probability changes near 0.5. In practice the splits they produce are nearly identical.

Question 6

How do you handle skewed features in a machine learning dataset, and why does skew matter?

Accepted Answer

Right-skewed features (long tail on the right) concentrate most values near zero while a few extreme values pull the mean up, which distorts distance-based models and linear regression. Common fixes are log, square-root, or Box-Cox transformations that compress the tail and make the distribution closer to normal, improving model convergence and reducing the undue influence of large values.

Question 7

How does k-means clustering work?

Accepted Answer

K-means partitions n points into k clusters by alternating between two steps: assigning each point to its nearest centroid, then recomputing each centroid as the mean of its assigned points. It repeats until assignments stop changing, which guarantees convergence but not a globally optimal solution.

Question 8

What is information gain and how does it relate to entropy in a decision tree split?

Accepted Answer

Information gain measures how much a split reduces uncertainty (entropy) in the target variable. It is the difference between the parent node's entropy and the weighted average entropy of the child nodes. The split that maximises information gain is selected at each node.

Question 9

How does k-nearest neighbours work, and why is it called a lazy learner?

Accepted Answer

KNN stores the entire training set and defers all computation to prediction time: for a new point it finds the k closest training examples by distance, then returns the majority class (classification) or mean value (regression). It is called lazy because there is no training phase — the model is the data itself.

Question 10

How does Naive Bayes work, and why is it called 'naive'?

Accepted Answer

Naive Bayes applies Bayes' theorem to classify by computing the posterior probability of each class given the features. It is naive because it assumes all features are conditionally independent given the class label — an assumption that is almost never true in practice, yet the classifier still works surprisingly well for text and other sparse data.

Question 11

When should you use one-hot encoding versus label encoding for categorical features?

Accepted Answer

Label encoding assigns each category an integer and implies an ordinal relationship that most algorithms will treat as meaningful distance. One-hot encoding creates a binary column per category and is correct for nominal data fed to linear or distance-based models. Use label encoding only when the category genuinely has an order, or when feeding tree-based models that handle it cleanly.

Question 12

What is the out-of-bag error in a random forest and how reliable is it as a validation estimate?

Accepted Answer

The OOB error is computed by predicting each training sample only with the trees that did not include it in their bootstrap sample. It is nearly unbiased and tracks closely with cross-validation accuracy, making it a free, practical validation estimate that does not require a separate hold-out split.

Question 13

What are overfitting and underfitting, and how do you fix each?

Accepted Answer

Overfitting occurs when a model memorizes training noise and fails to generalize; underfitting occurs when the model is too simple to capture the true signal. Fixes differ: overfitting requires regularization, more data, or reduced complexity; underfitting requires a more expressive model or better features.

Question 14

Why does R-squared always increase when you add features, and when should you use adjusted R-squared instead?

Accepted Answer

R-squared measures the proportion of variance explained by the model and can only increase or stay the same as features are added, even if those features are pure noise. Adjusted R-squared penalizes for the number of predictors, making it the right metric for comparing models with different numbers of features.

Question 15

What is the difference between standardization and normalization, and which models require feature scaling?

Accepted Answer

Standardization rescales features to zero mean and unit variance; normalization squashes values into a fixed range, usually [0, 1]. Distance-based and gradient-based models are sensitive to scale and require one of these; tree-based models split on rank order and are scale-invariant.

Question 16

What is the difference between supervised, unsupervised, and reinforcement learning?

Accepted Answer

Supervised learning trains on labeled input-output pairs to predict a target. Unsupervised learning finds structure in unlabeled data. Reinforcement learning trains an agent to maximize cumulative reward through trial-and-error interaction with an environment.

Question 17

Why do we split data into train, validation, and test sets, and what are the typical proportions?

Accepted Answer

The train set fits the model, the validation set tunes hyperparameters and guides model selection, and the held-out test set provides an unbiased estimate of final generalization error. Using the test set during development causes optimistic bias because the evaluation signal leaks into decisions.

Question 18

Why is linear regression unsuitable for binary classification, and what specific problems does logistic regression fix?

Accepted Answer

Linear regression predicts unbounded real values, so it can output probabilities below 0 or above 1, and its loss function penalizes confident correct predictions. Logistic regression fixes this by applying the sigmoid to map any real score to (0,1) and optimizing log-loss, which is a proper scoring rule aligned with probability calibration.

Question 19

What is the difference between bagging and boosting, and what error component does each primarily reduce?

Accepted Answer

Bagging trains many independent models on bootstrap samples in parallel and averages their predictions, primarily reducing variance. Boosting trains models sequentially, each correcting the errors of its predecessor, primarily reducing bias.

Question 20

What is the bias–variance tradeoff?

Accepted Answer

A model's expected test error splits into bias (error from over-simplified assumptions, causing underfitting), variance (sensitivity to the particular training sample, causing overfitting), and irreducible noise. Adding complexity lowers bias but raises variance, so the best model minimises their sum on unseen data — not the training error.

Question 21

How do you choose the number of clusters k in k-means?

Accepted Answer

The elbow method plots inertia against k and looks for the bend where adding another cluster gives diminishing returns. The silhouette score measures how similar each point is to its own cluster versus its nearest rival, with values closer to 1 indicating tighter, better-separated clusters. Both should be used together, not in isolation.

Question 22

What is the curse of dimensionality, and how does it affect machine learning models?

Accepted Answer

As the number of features grows, the volume of the feature space increases exponentially, so training data becomes exponentially sparse. Distance-based algorithms degrade because points become approximately equidistant; density estimation requires data that grows exponentially; and overfitting risk rises for any fixed training set size.

Question 23

What is data leakage in machine learning, and what are the most common ways it occurs?

Accepted Answer

Data leakage happens when information that would not be available at prediction time influences model training, producing overly optimistic evaluation metrics that collapse in production. Common sources include fitting preprocessors on the full dataset, including target-derived features, and using future data in time-series pipelines.

Question 24

What is pruning in decision trees and when would you use pre-pruning versus post-pruning?

Accepted Answer

Pruning removes splits that do not improve generalisation. Pre-pruning stops growth early via hyperparameters like max_depth or min_samples_leaf. Post-pruning (cost-complexity pruning) grows the full tree then collapses nodes whose removal does not hurt held-out accuracy enough.

Question 25

Walk me through exactly how a decision tree chooses a split at each node.

Accepted Answer

At each node the algorithm iterates over every feature and every candidate threshold, scores each candidate split by the weighted impurity of the two child nodes, and selects the pair that gives the largest impurity reduction. It then recurses on each child until a stopping criterion is met.

Question 26

What is the difference between discriminative and generative models, and when would you prefer each?

Accepted Answer

Discriminative models learn the conditional distribution P(y|x) directly and focus entirely on the decision boundary; generative models learn the joint distribution P(x,y) and can generate new samples. Discriminative models typically achieve higher classification accuracy with sufficient labeled data; generative models excel when data is scarce, you need to synthesize data, or the problem requires modeling the input distribution.

Question 27

How does early stopping work in gradient boosting, and why is it necessary?

Accepted Answer

Early stopping monitors a held-out validation metric after each tree is added and stops training when the metric has not improved for a given number of rounds. It is necessary because gradient boosting is not regularised by the number of trees alone — the training loss always decreases, but test loss will eventually increase.

Question 28

What problem does ElasticNet solve that neither Lasso nor Ridge can handle alone?

Accepted Answer

When predictors are highly correlated, Lasso tends to arbitrarily pick one and discard the others, producing unstable feature selection. Ridge retains all correlated features but cannot zero any out. ElasticNet combines both penalties to achieve stable, sparse solutions — it groups correlated features and can shrink the whole group together.

Question 29

What is the F1 score, why use the harmonic mean, and when is it the wrong metric?

Accepted Answer

F1 is the harmonic mean of precision and recall: 2PR/(P+R). The harmonic mean penalises extreme imbalance between the two — a model with 1.0 precision and 0.01 recall gets F1 = 0.02, not 0.505. F1 is the wrong metric when the classes are heavily imbalanced or when the costs of false positives and false negatives differ sharply, in which case F-beta, PR-AUC, or a cost-weighted metric is more appropriate.

Question 30

Why does regularization require feature scaling, and what happens if you skip it?

Accepted Answer

Regularization penalizes large coefficient magnitudes uniformly. If features are on different scales, a feature measured in thousands will naturally have a small coefficient while one measured in fractions will have a large one, so the penalty disproportionately shrinks some features and nearly ignores others. Standardization ensures the penalty is applied equally across all features.

Question 31

What are filter, wrapper, and embedded feature selection methods, and when do you use each?

Accepted Answer

Filter methods score features independently of the model using statistics like mutual information or correlation; they are fast but ignore feature interactions. Wrapper methods search subsets by actually training the model, finding better subsets at high computational cost. Embedded methods perform selection during training — LASSO and tree-based feature importances are the most common — offering a balance of quality and speed.

Question 32

Explain how gradient boosting fits residuals. What role does the learning rate play?

Accepted Answer

Gradient boosting builds an additive model by fitting each new tree to the negative gradient of the loss with respect to the current ensemble's predictions — effectively the residuals for squared error. The learning rate shrinks each tree's contribution, keeping the ensemble from over-correcting and acting as a regulariser.

Question 33

When should you use gradient descent over the normal equation to fit a linear regression?

Accepted Answer

The normal equation gives an exact closed-form solution in O(p³) time but becomes impractical when the number of features p is large (typically above ~10,000) because matrix inversion is cubic. Gradient descent scales as O(np) per iteration, making it the only viable option for large feature spaces or online learning.

Question 34

When should you use grid search vs random search vs Bayesian optimisation for hyperparameter tuning?

Accepted Answer

Grid search exhaustively tries every combination in a predefined grid, which is only practical for 1–2 hyperparameters. Random search samples combinations uniformly at random and finds good values faster per compute budget, especially when only a few hyperparameters actually matter. Bayesian optimisation fits a surrogate model of the objective and proposes the next trial intelligently, giving the best sample efficiency for expensive evaluations.

Question 35

How do decision trees and gradient boosting libraries handle categorical features natively, and when is label encoding safe?

Accepted Answer

sklearn trees require numeric input and treat label-encoded integers as ordinal, which imposes a false ordering. One-hot encoding is correct but expensive for high-cardinality features. XGBoost (v2+) and LightGBM support native categorical splits that find the optimal binary partition of categories without ordinal assumptions.

Question 36

How do you handle class imbalance in a machine-learning model?

Accepted Answer

Class imbalance is handled at the data level (oversampling with SMOTE, undersampling), the algorithm level (class weights, balanced bagging), and the decision level (threshold tuning). The right approach depends on how severe the imbalance is, how much data you have, and whether the minority class has sufficient local density to synthesise meaningfully. Always choose your evaluation metric first — accuracy is useless on imbalanced data.

Question 37

What are the strategies for handling missing values in a machine learning pipeline, and how do you choose between them?

Accepted Answer

Missing data can be dropped, imputed with a statistic (mean, median, mode), or imputed with a model. The right choice depends on the missing mechanism (MCAR, MAR, MNAR), the fraction of missing data, and the downstream model. Dropping rows is only safe when missingness is rare and random; imputation must always be fit on training data only.

Question 38

How do hierarchical clustering and DBSCAN differ from k-means?

Accepted Answer

Hierarchical clustering builds a tree of nested merges or splits and does not require specifying k upfront, but it is O(n² log n) and cannot revise early decisions. DBSCAN finds arbitrarily shaped clusters by density reachability, naturally marks outliers as noise, and also needs no k — but its results are sensitive to the eps and min_samples hyperparameters.

Question 39

How do you handle high-cardinality categorical features in machine learning?

Accepted Answer

One-hot encoding becomes impractical when a categorical feature has hundreds or thousands of unique levels, producing a sparse matrix that slows training and causes overfitting on rare categories. Better approaches include target encoding with smoothing, frequency encoding, hashing, learned embeddings, or grouping rare categories into an 'Other' bucket, each with different tradeoffs on leakage risk and information retention.

Question 40

How does a random forest work, and why does feature sampling at each split help more than row sampling alone?

Accepted Answer

A random forest grows many deep decision trees, each on a bootstrap sample of the rows, but also restricts each split to a random subset of features. Feature sampling decorrelates the trees so their errors cancel when averaged, which is the key source of variance reduction beyond what row sampling achieves.

Question 41

What is k-means++ and why is it better than random initialisation?

Accepted Answer

K-means++ initialises centroids by probabilistically spacing them apart: the first centroid is chosen uniformly at random, and each subsequent centroid is chosen with probability proportional to its squared distance from the nearest already-chosen centroid. This reduces the chance of bad starts, cuts the number of iterations to convergence, and provides an O(log k) approximation guarantee on the final inertia.

Question 42

What are the main limitations of k-means clustering?

Accepted Answer

K-means requires specifying k upfront, assumes clusters are convex and roughly equal in size and density, is sensitive to outliers and feature scale, and can converge to local minima. It struggles with non-globular shapes such as rings or crescents, and it assigns every point to exactly one cluster with no notion of uncertainty.

Question 43

How does the curse of dimensionality affect KNN?

Accepted Answer

In high-dimensional spaces all pairwise distances concentrate around the same value, so the concept of a 'nearest' neighbour breaks down — the k-th nearest neighbour is almost as far as every other point. KNN's accuracy degrades sharply as dimensionality increases unless the data has much lower intrinsic dimensionality.

Question 44

What is the fundamental difference between L1 (Lasso) and L2 (Ridge) regularization, and when do you choose each?

Accepted Answer

L1 adds the sum of absolute coefficient values to the loss, which drives some coefficients to exactly zero and performs implicit feature selection. L2 adds the sum of squared coefficients, which shrinks all weights proportionally but rarely zeroes any out. Lasso is preferred when you suspect only a few features matter; Ridge is preferred when most features contribute small effects.

Question 45

What are the core assumptions of linear regression, and what breaks when each is violated?

Accepted Answer

OLS linear regression rests on five assumptions: linearity, independence of errors, homoscedasticity, normality of residuals, and no perfect multicollinearity. Violating any one of them degrades coefficient estimates, standard errors, or the validity of hypothesis tests.

Question 46

What is log loss and why does it penalise confident wrong predictions more than uncertain ones?

Accepted Answer

Log loss (cross-entropy loss) measures how well a model's predicted probabilities match the true labels: it is the negative log-likelihood of the correct class. It penalises confident wrong predictions severely because log(p) approaches negative infinity as p approaches zero — predicting 0.99 for the wrong class incurs roughly 100x the penalty of predicting 0.6 for the wrong class. A perfect model achieves 0; a random binary classifier achieves ln(2) ≈ 0.693.

Question 47

What are the key regression metrics — MAE, RMSE, MAPE, R² — and what are their failure modes?

Accepted Answer

MAE, RMSE, MAPE, and R² each measure a different aspect of regression quality and each has a regime where it misleads. RMSE is dominated by outliers; MAE is robust but hides large-error tails; MAPE is undefined at zero and asymmetrically penalises under-prediction; R² can appear high even when absolute errors are large, and can be negative, yet is still commonly misread as a percentage-correct. Choosing the right metric requires knowing the cost structure of the prediction task.

Question 48

What is multicollinearity, how does it harm regression, and how do you detect and fix it?

Accepted Answer

Multicollinearity occurs when two or more predictors are highly linearly correlated, inflating the variance of coefficient estimates and making them numerically unstable and uninterpretable. The Variance Inflation Factor (VIF) quantifies how much each coefficient's variance is inflated relative to an orthogonal design.

Question 49

How does Ordinary Least Squares derive the coefficient vector, and what is the closed-form solution?

Accepted Answer

OLS minimizes the sum of squared residuals. Setting the gradient of the loss to zero yields the normal equations, whose unique solution is the projection of y onto the column space of X. The closed-form is the hat matrix formula β = (XᵀX)⁻¹Xᵀy.

Question 50

How do you detect and handle outliers in a machine learning dataset?

Accepted Answer

Outliers are detected via statistical rules (IQR, Z-score), visualization, or isolation-based algorithms. Handling options are removal, capping (Winsorization), transformation, or using robust algorithms. The right action depends on whether the outlier is a measurement error or a genuine extreme value — genuine extremes carry signal and should not be blindly removed.