Skip to content
datarekha

Course · 7 chapters · 39 lessons

Machine Learning

End-to-end ML workflow with scikit-learn, XGBoost, LightGBM, and the honest engineering practices that prevent silent failures in production.

The Machine Learning journey 0 / 39 completed
  1. Chapter 01

    Foundations

    7 lessons
  2. 01 What ML actually is Cut through the hype. Learn what machine learning actually does, how its three main settings differ, and when a plain rule is the better engineerin… Beginner5 min
  3. 02 The scikit-learn API The fit, transform, and predict contract makes scikit-learn models, preprocessing, validation, and deployment compose cleanly. Beginner6 min
  4. 03 Feature engineering & encoding On tabular data the model is a commodity — the features are the edge. Encoding categoricals, scaling numerics, interactions and aggregations, all w… Beginner8 min
  5. 04 Train/val/test & CV How to estimate performance on unseen data, choose honest splits, prevent leakage, and backtest models when rows are related or ordered in time. Beginner6 min
  6. 05 Bias–variance & learning curves A practical diagnostic for deciding whether bad validation performance comes from an underpowered model, an overfit model, weak data, or the wrong… Beginner7 min
  7. 06 Curse of Dimensionality Add features and your data gets exponentially lonelier. In high dimensions everything is far away, distances stop meaning anything, and intuition b… Intermediate8 min
  8. 07 Missing data and imputation Missing values are clues about the data-generating process, and safe imputation starts with that clue rather than a global average. Beginner15 min
  9. Chapter 02

    Regression

    3 lessons
  10. 08 Linear regression The model that started it all — and is still the right answer more often than people admit. Beginner8 min
  11. 09 L1, L2, Elastic Net Adding many features makes linear models overfit. Regularization is the fix you reach for before reaching for trees. Intermediate7 min
  12. 10 Gaussian processes A practical guide to function priors, kernel assumptions, predictive uncertainty, and the cubic-cost trade-off behind Gaussian processes. Advanced13 min
  13. Chapter 03

    Classification

    4 lessons
  14. 11 Logistic regression Despite the name, it's a classifier — and the workhorse classifier in industry. Outputs calibrated probabilities, not just labels. Beginner7 min
  15. 12 K-nearest neighbors A concrete guide to k-NN: how distance becomes a prediction, why scaling and the distance metric are part of the model, how k controls overfitting,… Beginner6 min
  16. 13 Naive Bayes Bayes' rule plus one bold assumption — that features are independent — gives a fast, surprisingly strong classifier, especially for text. Learn the… Beginner7 min
  17. 14 Support vector machines Find the boundary with the widest margin between classes, tolerate messy labels with soft margins, and bend a linear boundary with kernels. Intermediate8 min
  18. Chapter 04

    Trees & Boosting

    5 lessons
  19. 15 Decision trees The simplest non-linear model — interpretable, requires no scaling, handles mixed types. And the building block of every modern boosting library. Beginner7 min
  20. 16 Bagging, boosting & stacking Why a committee of models beats any single one. The unifying theory behind random forests and XGBoost — bagging cuts variance, boosting cuts bias,… Intermediate7 min
  21. 17 Random forests Take the high-variance decision tree, train hundreds of them on bootstrapped samples with random feature subsets, and average. That's it — and it w… Intermediate6 min
  22. 18 XGBoost, LightGBM, CatBoost Build trees sequentially, each correcting the last one's mistakes. The reason every winning Kaggle solution on tabular data uses XGBoost, LightGBM,… Intermediate9 min
  23. 19 SHAP & feature importance Built-in feature_importances_ is misleading. SHAP decomposes every prediction into per-feature contributions you can actually defend to a regulat… Advanced7 min
  24. Chapter 05

    Evaluation

    11 lessons
  25. 20 Metrics that matter Accuracy is a trap on imbalanced data. Here's the full crew — precision, recall, F1, ROC-AUC, PR-AUC, log-loss, MAE, RMSE, R² — and when to reach f… Intermediate7 min
  26. 21 Class imbalance When 99% of your rows are one class, "always predict negative" hits 99% accuracy and ships nothing useful. Here's how to actually fix it. Intermediate6 min
  27. 22 Data leakage The bug that makes your model look amazing in development and fail in production. Three forms, three fixes. Master the Pipeline pattern. Intermediate6 min
  28. 23 Hyperparameter tuning Grid search, random search, Bayesian search with Optuna. The pattern that prevents you from accidentally tuning on the test set. Intermediate7 min
  29. 24 Feature selection Choose a smaller, more useful set of input columns without leaking labels, missing interactions, or mistaking explanation for selection. Intermediate7 min
  30. 25 Model selection & nested CV If you tune hyperparameters and report the same CV score, you're lying to yourself. Nested cross-validation separates tuning from evaluation so you… Advanced7 min
  31. 26 Model Calibration When your model says 70%, does the thing happen 70% of the time? A model can rank perfectly (great AUC) and still lie about its probabilities. Cali… Advanced8 min
  32. 27 AutoML in practice AutoML can search models, preprocessing, hyperparameters, and ensembles in minutes. Learn how the search works, how to run an honest tabular baseli… Intermediate6 min
  33. 28 Conformal prediction Turn almost any point predictor into a finite-sample prediction interval or set with a clear coverage guarantee. Intermediate12 min
  34. 29 Survival analysis and time-to-event models Model when an event happens without pretending that unfinished follow-up means no event. Intermediate13 min
  35. 30 Learning to rank How search and recommendation systems learn order, from pairwise preferences to metric-aware reranking. Intermediate12 min
  36. Chapter 06

    Unsupervised

    6 lessons
  37. 31 K-means clustering A practical, from-first-principles guide to k-means: how assign-and-update minimizes squared distance, how initialization and scaling change the an… Beginner7 min
  38. 32 DBSCAN & hierarchical When k-means cuts winding shapes in half or forces outliers into a group, DBSCAN and hierarchical clustering offer different escapes: density-conne… Intermediate7 min
  39. 33 PCA & dimensionality reduction Understand how PCA rotates data onto its highest-variance directions, how to choose components, and when compression helps or quietly destroys usef… Intermediate8 min
  40. 34 t-SNE & UMAP Turn high-dimensional embeddings into useful 2D maps without fooling yourself. Understand the neighborhood-preserving mechanism, choose between t-S… Intermediate6 min
  41. 35 Gaussian mixture models A practical, probabilistic guide to Gaussian mixture models: soft cluster membership, covariance and density, EM fitting, model selection, producti… Advanced7 min
  42. 36 Anomaly detection Find rare, fraudulent, or broken events without reliable labels. Understand Isolation Forest, Local Outlier Factor, thresholds, and production fail… Intermediate7 min
  43. Chapter 07

    Responsible ML

    3 lessons
  44. 37 Interpretability: SHAP vs LIME Your model said no. Now explain why. The map of post-hoc interpretability — global vs local, LIME's local surrogate vs SHAP's additive Shapley valu… Advanced9 min
  45. 38 Fairness & bias in ML A model can be accurate overall and still systematically disadvantage a group. Learn where bias enters an ML system, how demographic parity, equali… Intermediate8 min
  46. 39 Causal inference for ML practitioners Learn when a model can predict an outcome and when a design or estimator is needed to answer what an intervention will change. Advanced13 min
  47. End of section 0 / 39 complete

    Make it stick — pass every quiz.

    Each lesson has a short quiz at the bottom. Passing the quiz is what marks the lesson complete and counts toward your certificate.

    Section complete 39 / 39 lessons

    Nice work — you finished Machine Learning.

    Certificates are earned per learning path, not per section. Here's where this section takes you:

FAQCommon questions

Machine Learning — frequently asked questions

Straight answers to the questions people ask most about machine learning.

Do neural networks always beat traditional ML?

No. On most tabular data, gradient-boosted trees (XGBoost, LightGBM, CatBoost) match or beat neural networks with less tuning and far less data. Deep learning dominates for images, text, and audio, but for structured business data, classical models are usually the stronger and simpler choice.

Read the lesson
What's the difference between training, validation, and test sets?

You fit the model on the training set, tune hyperparameters on the validation set, and report final performance once on the untouched test set. Keeping the test set truly held out is what makes your reported accuracy honest rather than optimistic.

Why is my model 99% accurate but useless?

Accuracy is misleading on imbalanced data — a model that always predicts the majority class can score 99% while catching none of the rare cases that matter. Use metrics suited to the problem, like precision, recall, F1, or AUC, and read the confusion matrix.

Read the lesson
What is data leakage and why is it dangerous?

Data leakage is when information unavailable at prediction time sneaks into training — like scaling using the whole dataset before splitting, or a feature derived from the target. It produces great validation scores that collapse in production, making it one of the most costly silent bugs in ML.

Read the lesson
When should I use XGBoost versus a random forest?

Random forests are robust and need little tuning, so they make a strong baseline. Gradient boosting (XGBoost and friends) usually reaches higher accuracy but is more sensitive to hyperparameters and overfitting. Start with a random forest, then try boosting to squeeze out more performance.