Broadcasting

The last lesson kept teeing this up: add a length-1 axis “so it will broadcast,” and promised the rule was coming. Here it is. Broadcasting is how NumPy combines arrays of different shapes — adding a vector to every row of a matrix, scaling each column, filling a grid from a column and a row — with no explicit loop written anywhere. Once it clicks, it rewires how you write array code: you stop thinking in loops and start thinking in shapes.

The rule

NumPy aligns the shapes right-to-left (trailing axis first, then working inward). For each axis:

• if the sizes are equal → fine
• if one of them is 1 → it stretches to match the other
• otherwise → error

That is the entire rule. Right-to-left alignment is the convention because array elements are stored row-by-row in memory — the last axis varies fastest, so it is the natural anchor for matching shapes.

The 3 patterns you’ll use 90% of the time

1. Subtract a row from every row (centering)

import numpy as np

# 4 samples, each with 3 features.
X = np.array([
    [10, 200, 30],
    [12, 210, 32],
    [11, 205, 29],
    [13, 215, 31],
], dtype=float)

# The per-column mean — shape (3,)
mean = X.mean(axis=0)
print("mean:", mean)
print("mean shape:", mean.shape, "— X shape:", X.shape)

# Subtract the mean from every row, in one operation:
X_centered = X - mean
print(X_centered)

mean: [ 11.5 207.5  30.5]
mean shape: (3,) — X shape: (4, 3)
[[-1.5 -7.5 -0.5]
 [ 0.5  2.5  1.5]
 [-0.5 -2.5 -1.5]
 [ 1.5  7.5  0.5]]

The mean has shape (3,), X has shape (4, 3). NumPy lines them up as (1, 3) vs (4, 3) — the 1 broadcasts down all four rows, and the subtraction runs on every row at once. This is exactly how you “center” features before PCA or linear regression. No loop required.

2. Scale each column independently

import numpy as np

X = np.array([
    [1, 100, 0.01],
    [2, 200, 0.02],
    [3, 300, 0.03],
], dtype=float)

# Each column has wildly different scale. Divide by per-column max.
scale = X.max(axis=0)        # shape (3,)
X_scaled = X / scale         # broadcasts (3,) across rows
print(X_scaled)

[[0.33333333 0.33333333 0.33333333]
 [0.66666667 0.66666667 0.66666667]
 [1.         1.         1.        ]]

3. Outer product — column × row

import numpy as np

# A column vector (3, 1) times a row vector (1, 4)
col = np.array([[1], [2], [3]])       # shape (3, 1)
row = np.array([[10, 20, 30, 40]])    # shape (1, 4)

print("col shape:", col.shape, "row shape:", row.shape)
print("col * row:")
print(col * row)        # broadcasts to (3, 4)

col shape: (3, 1) row shape: (1, 4)
col * row:
[[ 10  20  30  40]
 [ 20  40  60  80]
 [ 30  60  90 120]]

This is the outer product pattern: the (3, 1) column stretches across 4 columns, the (1, 4) row stretches down 3 rows, and every cell is the product. You meet it in attention mechanisms, in pairwise distances, and in countless physics simulations.

When broadcasting fails

import numpy as np

A = np.zeros((3, 4))
v = np.array([1, 2, 3])      # shape (3,) — trailing axis is 3, not 4

try:
    A + v
except ValueError as e:
    print("Failure:", e)

# Fix: explicitly make v a column.
v_col = v[:, np.newaxis]     # shape (3, 1)
print((A + v_col).shape)     # (3, 4) ✓

Failure: operands could not be broadcast together with shapes (3,4) (3,) 
(3, 4)

The (3,) vector aligns its trailing axis (3) against A’s trailing axis (4) — 3 ≠ 4, and neither is 1, so it fails. Make v a (3, 1) column with v[:, np.newaxis] (or v.reshape(-1, 1)) and now the trailing 1 stretches across the 4 columns: (3, 1) + (3, 4) → (3, 4). That one move fixes 80% of “could not broadcast” errors.

A non-obvious use: distance matrix

Compute pairwise squared distances between every pair of points — with no loop at all:

import numpy as np

# 4 points in 2D
pts = np.array([
    [0.0, 0.0],
    [1.0, 0.0],
    [0.0, 1.0],
    [3.0, 4.0],
])

# pts[:, None, :] shape (4, 1, 2)
# pts[None, :, :] shape (1, 4, 2)
# Difference broadcasts to (4, 4, 2)
diff = pts[:, None, :] - pts[None, :, :]
dist_sq = (diff ** 2).sum(axis=-1)
print(dist_sq.round(2))

[[ 0.  1.  1. 25.]
 [ 1.  0.  2. 20.]
 [ 1.  2.  0. 18.]
 [25. 20. 18.  0.]]

The (4, 1, 2) and (1, 4, 2) arrays broadcast to a (4, 4, 2) cube of every pairwise difference, and summing the last axis collapses it to a 4 × 4 distance matrix (symmetric, zero on the diagonal). What you just wrote is the inner loop of k-nearest-neighbors — the “two new-axis insertions” pattern is the cleanest way to do all-pairs operations in NumPy.

In one breath

Broadcasting combines different-shaped arrays without a loop by aligning shapes right-to-left and, per axis, requiring equal sizes or a 1 that stretches to match (anything else errors). It powers the everyday patterns: subtract a (3,) per-column mean from a (4, 3) matrix (centering), divide by a per-column max (scaling), and multiply a (3, 1) column by a (1, 4) row into a (3, 4) outer product. A (3,) vector won’t add to a (3, 4) matrix (trailing 3 ≠ 4) until you make it a (3, 1) column with np.newaxis — the fix for most “could not broadcast” errors. Two new-axis insertions, pts[:, None] - pts[None, :], build an all-pairs distance matrix — the kNN inner loop in one line.

Practice

A question to carry forward

Broadcasting answered only half the question — it decides the shape of the result, how a (4, 3) and a (3,) line up into (4, 3). But it never said what actually computes each output number. When you wrote X - mean or col * row, some machinery applied the subtraction and the multiplication element by element, at C speed, with no Python loop in sight.

Those element-wise engines — +, *, np.exp, np.sqrt, np.maximum — are universal functions, or ufuncs, and broadcasting is really just the rule for how a ufunc lines its inputs up before it runs. What is a ufunc, why is it so much faster than the equivalent Python loop, and what do its hidden controls — out= to compute in place without allocating, and .reduce / .accumulate to fold an array down — let you do?