K-means clustering

Your product team has 50,000 customers and no segment labels. You have two useful measurements for each one: monthly spend and number of orders. Marketing asks for three groups, but nobody has labelled which customers are “casual”, “regular”, or “premium”.

You could draw a line through the chart by eye. That stops being charming at 50,000 rows.

K-means is an unsupervised learning algorithm: it groups data without a target label. You choose the number of groups, k, and k-means finds k representative points called centroids. Each observation joins the nearest centroid.

The result is not a truth hiding in the database. It is a partition produced by a particular distance measure, scaling choice, and value of k. Those details decide the answer.

The mental model

Imagine dropping three magnets onto a scatter plot. Every point belongs to the magnet that is closest. Each magnet then slides to the average location of the points it owns. Repeat.

The magnets are centroids. “Average location” means the arithmetic mean of every feature, coordinate by coordinate. Points at (2, 4) and (6, 8) have centroid (4, 6).

K-means uses Euclidean distance. In two dimensions, the squared distance between (x₁, y₁) and (x₂, y₂) is:

(x₁ - x₂)² + (y₁ - y₂)²

For comparing centroids, the square root is unnecessary because squaring preserves which one is nearest. K-means specifically minimizes squared Euclidean distance; that objective is why the arithmetic mean is the update. Ordinary distance would be a different objective with a different minimizing update.

The quantity being minimized is inertia:

inertia = sum of squared distances from points to their assigned centroids

Low inertia means points are close to their assigned centres, but it does not prove the grouping is meaningful. With one cluster per customer, inertia is zero.

A complete small example

These four customers already have features on comparable scales:

Ask for k = 2 and start with:

• C₁ = (1, 1)
• C₂ = (9, 7)

These happen to be customers A and D. That is valid initialization, not a requirement.

Round one: assign

Compare each point with both centroids.

For A:

• to C₁: (1 - 1)² + (1 - 1)² = 0
• to C₂: (1 - 9)² + (1 - 7)² = 100

A joins cluster 1.

For B:

• to C₁: (2 - 1)² + (1 - 1)² = 1
• to C₂: (2 - 9)² + (1 - 7)² = 85

B joins cluster 1.

For C:

• to C₁: (8 - 1)² + (8 - 1)² = 98
• to C₂: (8 - 9)² + (8 - 7)² = 2

C joins cluster 2. D joins cluster 2 because it is already at C₂.

The assignments are:

• cluster 1: A and B
• cluster 2: C and D

Round one: update

Move each centroid to the mean of its assigned points:

C₁ = ((1 + 2) / 2, (1 + 1) / 2) = (1.5, 1)

C₂ = ((8 + 9) / 2, (8 + 7) / 2) = (8.5, 7.5)

The points did not move; the centroids did. The new inertia is:

• A to (1.5, 1): 0.25
• B to (1.5, 1): 0.25
• C to (8.5, 7.5): 0.5
• D to (8.5, 7.5): 0.5

Total inertia: 1.5, down from the initial 3.

Round two: assign again

The assignments stay the same. A point at (5, 4), for example, would have squared distances:

• to (1.5, 1): 21.25
• to (8.5, 7.5): 24.5

It would join cluster 1 even though it sits between the groups. K-means always assigns every point; it has no built-in “neither” option.

The causal mechanism is:

1. With centroids fixed, nearest-centroid assignment minimizes each point’s contribution to inertia.
2. With assignments fixed, the arithmetic mean minimizes the cluster’s sum of squared distances.

Each step therefore improves, or leaves unchanged, the same objective. Inertia cannot increase from one round to the next. The algorithm stops when assignments stop changing, centroids move less than a tolerance, or a maximum iteration count is reached.

This guarantees only a local minimum: a solution that cannot improve through the next assign-and-update moves. It is not guaranteed to find the best arrangement of k clusters. Different starting centroids can produce different final answers.

What the geometry assumes

Each centroid owns the region containing points closer to it than to any other centroid. The borders are straight lines in two dimensions, or flat hyperplanes in higher dimensions. This favours groups that are roughly compact, convex, and similar enough in size and density.

That geometry fails for interleaved crescents and nested rings, and a large sparse group can pull its centroid away from a dense compact group. Outliers are especially influential because squared distance magnifies them: a point ten units away contributes 100, while one two units away contributes 4.

Scale is part of the model

Suppose monthly spend ranges from 0 to 2,000 dollars and monthly orders from 0 to 20. A difference of $100 and one order contributes:

100² + 1² = 10,001

Spend dominates because of its units. K-means is obeying the distance question you supplied.

A common fix is standardization: subtract each feature’s mean and divide by its standard deviation. A one-standard-deviation difference in spend then has the same numerical weight as one in orders. Use this only when those weights fit the question. If spend should count ten times as much, standardizing removes a business decision you intended to keep.

Fit the scaler on reference data and reuse those parameters for new observations. Recomputing them changes the coordinate system and makes clusters across time harder to compare. For skewed features or extreme values, consider log1p, clipping, or robust scaling.

Do not encode categories as arbitrary integers and use Euclidean distance: assigning red, blue, and green the values 1, 2, and 3 invents distances that usually mean nothing. Encode features deliberately or use a method suited to mixed data. In high dimensions, distances can also become less distinctive; remove irrelevant features or use a justified representation.

Initialization: why k-means++ helps

A poor start can put two centroids in one dense patch and leave another without one. Lloyd’s local updates may then settle into a lopsided partition.

K-means++ spreads starting centroids by choosing the first centre, then giving each point a probability proportional to the squared distance from its nearest chosen centre. Far-away regions are more likely to receive the next centre.

It usually gives better starts than uniform random selection, but it is not a guarantee. Use multiple restarts: run the algorithm from several initial configurations and retain the result with lowest inertia. In scikit-learn, n_init=10 means ten complete runs, not ten extra update rounds. max_iter controls the length of one run; n_init controls how many starts you try.

Choosing k without pretending the elbow is a law

K-means cannot discover the number of clusters. You provide k.

The elbow method fits several values of k and plots inertia. The best achievable inertia cannot increase as k grows, though it may stay flat; at one cluster per point, it is zero. Look for where improvement becomes marginal, but real curves are often smooth and may have no elbow.

The silhouette score compares a point’s average distance to its own cluster, a(i), with its lowest average distance to another cluster, b(i):

(b(i) - a(i)) / max(a(i), b(i))

The score ranges from -1 to 1. Values near 1 indicate separation, near 0 a boundary, and below 0 a possible misassignment. Silhouette inherits k-means’ distance geometry, so it can favour compact blobs even when the real structure is non-convex.

Choose k using:

• elbow and silhouette trends
• stability across restarts and resampled data
• actionable cluster sizes
• differences that matter to the decision

A slightly weaker score can be worthwhile if the resulting group supports a useful intervention. Cluster IDs are arbitrary; inspect features before naming groups.

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from sklearn.metrics import silhouette_score

X, _ = make_blobs(
    n_samples=600,
    centers=3,
    cluster_std=1.1,
    random_state=0,
)

print(f"{'k':>2} {'inertia':>10} {'silhouette':>11}")
for k in range(2, 7):
    model = KMeans(
        n_clusters=k,
        n_init=10,
        random_state=0,
    )
    model.fit(X)
    score = silhouette_score(X, model.labels_)
    print(f"{k:2d} {model.inertia_:10.0f} {score:11.3f}")

The production pattern

During exploration, fit several values of k and inspect profiles and stability. In production, freeze preprocessing and centroids, then assign new observations to the nearest existing centroid.

from sklearn.cluster import KMeans
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

model = make_pipeline(
    StandardScaler(),
    KMeans(n_clusters=3, n_init=10, random_state=0),
)

model.fit(customers_reference)
new_cluster_ids = model.predict(customers_new)

The reference and new data must have the same feature columns in the same order. Monitor cluster counts, feature distributions, nearest-centroid distances, and assignment shares. A rise in nearest-centroid distance can signal that new data no longer resembles the fitted data. For novelty or outlier detection rather than forced assignment, see anomaly detection.

Failure modes you can see first

Assignments change on every run. Compare memberships with a label-invariant measure such as adjusted Rand index; cluster IDs can change through a harmless permutation. Genuine instability may indicate overlap, a poor k, unstable features, or too few restarts. Check lowest-inertia results and resampled fits.

A tiny cluster contains bizarre records. Inspect outliers and data errors. K-means is sensitive to extreme values because of squared distance; it is not outlier-robust.

The silhouette is poor and the plot looks sliced. A score near zero or negative values, with one visual shape cut into pieces, indicates a geometry mismatch. More iterations will not fix it. Try DBSCAN or hierarchical clustering for non-convex or density-shaped groups, or Gaussian mixture models for soft ellipsoidal membership.

The clusters are polished but unusable. K-means optimizes compactness, not revenue, fairness, causal impact, or campaign response. Profile the groups, test stability, and measure the downstream decision. If the goal is to predict a known outcome, use a supervised model; clustering does not establish causation.

K-means is useful for many numerical observations, a meaningful distance, reasonably compact groups, and fast fitting or assignment. It is the wrong geometry for crescents, rings, uneven density, severe outliers, and many mixed categorical datasets. Sometimes the most honest result is that the data has no useful cluster structure.

In one breath

K-means alternates nearest-centroid assignment with moving each centroid to the mean of its assigned points. The assignment step is optimal for fixed centroids, and the mean is optimal for fixed assignments, so inertia cannot increase. The result can still be a local rather than global solution, making k-means++ and multiple restarts important.

Scaling changes the distance question. Use the elbow as a guide, silhouette as a geometry-dependent measure, and stability, cluster profiles, sizes, and downstream usefulness as checks. K-means is fast and effective for compact numerical groups, but it always returns k clusters—even when no useful clusters exist.

Quick check

Next

When clusters have irregular shapes or varying density, DBSCAN & hierarchical clustering offers better tools. When the feature space is wide, PCA can provide a useful, carefully interpreted reduction before you inspect or cluster it.