What ML actually is

At 9:07 on Monday morning, your inbox receives 40,000 messages. Some are invoices. Some are messages from your manager. Some claim you have won a yacht despite never entering a yacht competition.

Nobody wrote a rule for every possible scam. A new sender, a misspelled brand, a shortened link, and a suspicious attachment can appear in combinations nobody anticipated. Yet an email system can still estimate which messages are probably spam because it has seen many earlier messages labelled “spam” and “not spam”.

That is the central problem machine learning solves: software must make useful predictions when the rules are too messy, numerous, or context-dependent to write by hand.

A traditional program takes data and rules that a person wrote, then produces an answer:

input + hand-written rules → output

Machine learning reverses the difficult part:

examples of inputs and answers → learned model

The learned model then handles new inputs:

new input + learned model → prediction

A model is the learned mathematical object that turns inputs into predictions. It might be a set of equations, a decision tree, thousands of small trees, or billions of neural-network parameters. “Learned rules” is useful shorthand, but do not picture a tiny programmer typing readable if statements. Most models discover numerical patterns, not explanations a human would have chosen.

The mental model: examples become a prediction machine

Take 10,000 past emails. For each, record measurable properties and a label supplied by a person or an existing process.

An individual measurable property is a feature. For our toy spam system, features might include:

• the number of links;
• the age of the sender’s account;
• whether the word “free” appears.

The answer the system should learn is the label, also called the target. Here it is spam or not spam. A collection of examples containing features and labels is a labelled dataset.

For this toy supervised model, training has four pieces:

1. The model starts with adjustable numbers called parameters.
2. It makes predictions from the features.
3. A loss function measures how wrong those predictions are.
4. A training algorithm changes the parameters to reduce the loss across many examples.

After training, the parameters are fixed. New email passes through the model without changing it. This stage is called inference.

Here is a deliberately small model so the arithmetic is visible. It calculates a score:

score = -2 + 0.7 × links - 0.8 × sender_age_years + 0.9 × free_word

The four numbers are parameters. The negative -2 is an intercept, or starting point. The other numbers are weights, which say how strongly each feature moves the score. A positive weight raises the spam score; a negative weight lowers it.

The model converts the score into a probability with a sigmoid, which squeezes any number into the range from zero to one:

p(spam) = 1 / (1 + e^(-score))

Consider two messages.

from math import exp

messages = {
    "A": (3, 0.1, 1),
    "B": (0, 5, 0),
}

for name, (links, sender_age, free_word) in messages.items():
    score = -2 + 0.7 * links - 0.8 * sender_age + 0.9 * free_word
    probability = 1 / (1 + exp(-score))
    print(f"{name}: score={score:.2f}, spam probability={probability:.3f}")

For message A, the score is -2 + 0.7 × 3 - 0.8 × 0.1 + 0.9 × 1, which equals 0.92. The sigmoid turns that into roughly 0.715, or a 71.5 percent estimated chance of spam.

For message B, the score is -2 + 0 - 0.8 × 5 + 0, which equals -6. Its estimated chance is roughly 0.2 percent.

The model did not learn that “three links always means spam”. It combined several weak clues. A legitimate newsletter might have many links but come from a sender with a long history; a scam might have one link and a brand-new account.

During training, the model compares predictions with known labels and adjusts its weights. If it assigns a high spam probability to a message labelled “not spam”, the loss increases. Repeated errors push the parameters toward patterns that better fit the labels.

This is why the data matters more than the word “AI”. If the labels say every company newsletter is spam, the model may faithfully learn that association. It has no private access to the meaning of “legitimate”.

Training accuracy is not the goal

A model can perform perfectly on examples it has already seen by memorising them. That is overfitting: capturing accidental details instead of patterns that work on new data. The property we want is generalisation, or good performance on unseen cases.

Reserve labelled examples for a training set, validation set, and test set before fitting anything. Fit parameters on training data; use validation data to compare models, choose hyperparameters, and select a decision threshold; evaluate once on the untouched test set. Cross-validation can repeatedly divide the training portion into folds to make model selection more stable, but the final test set must remain outside that process.

A probability is not yet a product decision. If the model says p(spam) = 0.715, the application still needs a policy: delete the message, place it in a folder, show a warning, or do nothing. The threshold should reflect costs. Hiding a genuine invoice may be worse than allowing one more scam into the inbox.

The three main learning settings

The categories describe where the learning signal comes from.

Supervised learning: learn from answer keys

Supervised learning uses examples where the desired answer is known. When the target is a category, the task is classification: spam versus not spam is binary classification, while choosing among five product categories is multiclass classification.

When the target is a number, the task is regression: predicting a delivery time, house price, or number of support tickets. Supervised learning is common in business because the question is explicit: “Given what we know now, what outcome happened later?”

The catch is that labels cost time and money. Someone must decide what counts as spam, a late delivery, or fraud. The label must also match the decision: predicting “customer eventually complained” may be a poor substitute for “chargeback within 60 days”.

Unsupervised learning: find structure without an answer key

Unsupervised learning receives inputs without a target label. It looks for groups, low-dimensional structure, or unusual observations.

With email features but no spam labels, clustering might discover groups of short messages from new senders, newsletters, and account notifications. That can organise an inbox or guide a labelling project, but it does not prove that one group is spam.

Anomaly detection similarly marks observations unlike the usual population. “Unusual” is not identical to “bad”: a legitimate bulk purchase can be anomalous. A cluster is a description of the data, not a natural law; feature scaling and the chosen number of clusters can change the result.

Reinforcement learning: learn through consequences

Reinforcement learning trains an agent that chooses actions inside an environment. The agent observes a state, takes an action, receives a reward, and tries to improve its future total reward. A policy maps states to actions.

Imagine a warehouse robot choosing its speed every second. Moving quickly rewards a faster delivery; colliding with a shelf causes a large penalty. An action can change what happens later, creating a credit-assignment problem: which earlier action deserves blame or credit for a delayed reward?

Online reinforcement learning may also need exploration, trying uncertain actions to discover whether they are better. In a live warehouse that can be dangerous, so practical systems use offline logs, simulation, safety constraints, and human overrides. A spam classifier has direct historical labels and does not need to experiment on unsuspecting users.

A modern fourth term: self-supervised learning

Self-supervised learning creates targets from the data itself. A language model might hide a word and predict it from surrounding text; an image system might learn whether two altered views came from the same image.

The targets are generated rather than hand-written, but training still compares predictions with targets. Self-supervision is best understood as a way to manufacture a learning signal. Large language models use it to learn broad statistical patterns from text, then may be adapted with supervised examples or preference signals. Pretraining does not guarantee current truth, perfect instruction-following, or suitability for high-stakes decisions.

Choosing ML instead of a rule

Start with the decision, not the fashionable model.

For “flag every transaction over $10,000”, use amount > 10000, not a classifier trained on yesterday’s transactions. The rule is exact, cheap, fast, easy to audit, and stable. A model would add uncertainty without useful information.

ML earns its complexity when the rule is difficult to state. “Flag transactions that look unlike legitimate behaviour” may depend on amount, merchant, geography, timing, device, and account history. Even then, hard safety rules, human review, rate limits, and audit logs may still be necessary.

Do not treat lots of data as permission to use ML. A million examples with unreliable labels can be worse than 10,000 carefully checked ones. Data needs depend on pattern complexity, feature count, target noise, and the performance required.

ML also brings continuing costs:

• collecting and defining labels;
• building a reliable feature pipeline;
• serving predictions within the required latency;
• monitoring quality and retraining after behaviour changes;
• investigating unfair outcomes and protecting sensitive data.

A hand-written rule can be wrong, but its failure is visible in the rule itself. A model can fail quietly after deployment. ML is a poor fit when mistakes are catastrophic and there is no meaningful review or fallback, when the target changes faster than labels can be collected, or when every decision needs a clear explanation.

Failure modes you will actually see

Data leakage occurs when information unavailable at prediction time, information from the answer itself, or evaluation data enters the model or evaluation process. For example, a feature called user_moved_to_spam_folder records an action taken after the user saw the email. Offline performance may look excellent, but the feature is unavailable when serving the prediction. Construct features using only information available at that moment, split related records carefully, and keep the test set untouched.

Class imbalance can make accuracy meaningless. If 9,950 of 10,000 test emails are legitimate, a system that calls everything legitimate achieves 99.5 percent accuracy while catching zero spam. Inspect the confusion matrix and use metrics that match the harm: recall asks what fraction of spam was caught; precision asks what fraction of flagged messages was truly spam.

Distribution shift occurs when the world changes: spammers alter vocabulary, infrastructure changes, or users move to different devices. Monitor input distributions, delayed outcome metrics, and performance across important groups. A running dashboard is not proof that the model remains useful.

Feedback loops arise when the model changes the data it later learns from. If automatically blocked messages disappear from future training data, the model sees a distorted sample. Keep audited samples of incoming data, review borderline cases, and preserve a correction path.

ML, AI, and the current vocabulary

Artificial intelligence is the broad umbrella for systems performing tasks associated with intelligence, including search, planning, rules, optimisation, and machine learning.

Machine learning learns patterns from data. Deep learning uses neural networks with many learned layers. Generative AI produces new content such as text, images, audio, or code. A large language model is a generative language model, usually trained through self-supervised pretraining and additional adaptation.

A product called an “AI agent” may combine an LLM, ordinary code, database queries, retrieval, and deterministic safety checks. Ask what data trains it, what output it produces, what action follows, and what happens when it is wrong.

In one breath

Machine learning fits a model from examples so software can make predictions on new inputs. A feature is an input measurement; a label is the known answer; a loss measures error; training adjusts parameters; inference uses the finished model.

Supervised learning uses labelled examples. Unsupervised learning finds structure without answer labels. Reinforcement learning learns actions from rewards over time. Self-supervised learning creates targets from raw data.

Use ML when the target is clear, the pattern is hard to express as rules, the data reflects the real decision, and mistakes can be managed. Use a rule when the policy is exact. Labels, evaluation, monitoring, and safeguards matter as much as fitting the model.

Quick check

What’s coming in this section

The next lessons make the training loop practical: the scikit-learn API, reliable train/test splitting, regression, classification, trees, and ensembles. They inherit the same responsibility introduced here: define the target carefully, test on genuinely new data, and know what a wrong prediction costs.