Confusion Matrix, Precision, Recall, ROC

Last lesson ended with the indictment: on the 5% positive medical set, a model that predicts “negative” for everyone scores 95% accuracy and catches not one sick patient. Accuracy lied because it lumped every error into one heap. To see which kind of mistake a model makes, you have to pull that heap apart — and a single number never can.

Think about what is really at stake. A model that flags a healthy patient as sick raises a false alarm; a model that waves a sick patient through commits a miss. Those two failures cost wildly different things — and accuracy, which only counts total rights and wrongs, cannot tell them apart. What can is a small table that cross-tabulates what the model said against what was true. That table is the confusion matrix, and precision, recall, F1, and AUC all read straight off its four cells.

The confusion matrix

Cross “what the model predicted” against “what was actually true.” Each of the four cells counts one outcome:

• TP — predicted positive, actually positive (a correct hit).
• FP — predicted positive, actually negative (a false alarm).
• FN — predicted negative, actually positive (a miss).
• TN — predicted negative, actually negative (a correct rejection).

The four metrics

From those four counts come the four numbers GATE asks for:

• Accuracy = (TP + TN) / total — the fraction of all predictions that were correct. The number that lied last lesson.
• Precision = TP / (TP + FP) — of everything flagged positive, how many really were. High precision means few false alarms.
• Recall (sensitivity, TPR) = TP / (TP + FN) — of everything actually positive, how many you caught. High recall means few misses.
• F1 = 2 · P · R / (P + R) — the harmonic mean of precision and recall. It stays low unless both are high, so it folds the trade-off into one number.

Drag the decision threshold and watch the four cells — and therefore every metric — move together:

ROC and AUC

A classifier really outputs a score, and you pick a threshold to turn it into a yes/no. Sweep that threshold from strict to lenient and plot TPR (recall, TP/(TP+FN)) on the y-axis against FPR (FP/(FP+TN)) on the x-axis — the traced curve is the ROC curve. AUC, the area under it, is the probability that the model ranks a random positive above a random negative: 0.5 is coin-flip guessing, 1.0 is perfect. Because AUC never fixes a single threshold, it is a clean one-number comparison of two models’ ranking ability.

How GATE asks this

The reliable NAT hands you a populated confusion matrix and asks for one metric — precision, recall, F1, or accuracy — to two or three decimals. The recipe never varies: read off TP, FP, FN, TN, then plug into the ratio. The MSQ variant builds the same counts from a word problem and asks which statements hold — GATE DA 2026 (Q47) gave one class 20 items and the other 10, with a handful misclassified each way, then asked you to compare the two classes’ accuracy, precision, and recall. Either way you must keep the precision-vs-recall denominators straight, and remember that accuracy lies under class imbalance.

Worked example — GATE DA 2026

A binary classifier on 30 samples produces TP = 8, FP = 2, FN = 6, TN = 14. Compute accuracy, precision, recall, and F1.

Check the total first: 8 + 2 + 6 + 14 = 30. Then take each metric in turn:

accuracy  = (TP + TN) / total = (8 + 14) / 30 = 22/30  ≈ 0.733
precision = TP / (TP + FP)    = 8 / (8 + 2)   = 8/10   = 0.800
recall    = TP / (TP + FN)    = 8 / (8 + 6)   = 8/14   ≈ 0.571
F1        = 2·P·R / (P + R)   = 2·0.8·0.571 / (0.8 + 0.571)
          = 0.914 / 1.371                              ≈ 0.667

So accuracy ≈ 0.733, precision = 0.80, recall ≈ 0.571, F1 ≈ 0.667. Precision (0.80) beats recall (0.571), exactly as the few-false-alarms / more-misses cell counts predicted: the model is cautious — when it says positive it is usually right, but it still misses 6 of the 14 actual positives.

In one breath

The confusion matrix cross-tabulates predictions against truth into four counts — TP, FP, FN, TN — and every metric is a ratio of them: accuracy (TP+TN)/total counts all correct calls (but lies under imbalance), precision TP/(TP+FP) is how many flagged positives are real, recall TP/(TP+FN) is how many real positives are caught, and F1 2PR/(P+R) is their harmonic mean; sweeping the score threshold traces the ROC curve of TPR against FPR, whose area AUC is the threshold-free chance the model ranks a random positive above a random negative.

Practice

A question to carry forward

Every metric here — the confusion matrix, precision, recall, the ROC curve — quietly assumes you already have a classifier: something that emits “positive” or “negative,” or better, a score you can threshold. We have spent six lessons building models, but every one of them predicted a number. Not once have we built a machine that outputs a class.

So the obvious gap opens. The ROC curve practically begs for it — it sweeps the threshold on a model’s score, treating that score as a probability of being positive. Where does such a probability come from? Here is the thread onward: can you take the linear machinery of regression — that same wᵀx + b weighted sum — and bend its unbounded output into a clean probability between 0 and 1, giving you the first true classifier, the one every metric in this lesson was waiting to grade?