Why linear algebra is the language of ML

Open any machine-learning paper and the first equation you meet looks like y = Xw + b. Open the code behind that same paper and the first line is import numpy as np. Those two openings are not cousins — they are the same sentence written twice, once in mathematics and once in code. Linear algebra is the notation; NumPy is that notation made runnable.

That is why this chapter comes first. You do not need a semester of proofs to read either line. You need to recognise four shapes of operation and know, for each one, what it does to your data. Once you can see those four shapes, the models later in this course stop looking like a wall of symbols and start looking like what they are: data flowing through a small handful of moves you already understand.

Data is a matrix

Begin with the thing you actually hold — a dataset. Lay it out as a table and a shape appears on its own: a dataset with n samples and d features is a grid of numbers, n rows tall and d columns wide. That grid has a name. We call it a matrix of shape (n, d).

Pull one row out of that grid and you have a single sample — a list of d numbers, a vector of length d. A model, then, is nothing more exotic than a function that takes such a vector and hands back a prediction. Hold those two words steady — matrix for the dataset, vector for the one sample — because everything that follows is just operations performed on these two shapes.

The four operations

Four moves do almost all the work. Watch what each one takes in and what it gives back, and a surprising amount of ML stops being mysterious.

1. Vector · vector → a number (similarity)

The dot product of two vectors of the same length multiplies them position by position and adds the results. What comes out is not another vector but a single scalar — one plain number:

a · b = a[0]·b[0] + a[1]·b[1] + … + a[d-1]·b[d-1]

That number has a meaning you can feel: it measures how aligned the two vectors are. A big positive value means they point much the same way; zero means they are perpendicular; a negative value means they pull in opposite directions.

import numpy as np

# Three users' taste vectors over [action, comedy, drama]
alice = np.array([0.9, 0.1, 0.4])
bob   = np.array([0.8, 0.2, 0.3])
eve   = np.array([0.1, 0.9, 0.0])

# A higher dot product means more aligned taste
print("alice · bob:", round(float(alice @ bob), 2))
print("alice · eve:", round(float(alice @ eve), 2))

alice · bob: 0.86
alice · eve: 0.18

Alice and Bob both love action, so their vectors line up and the dot product is large; Eve lives in comedy, so hers barely overlaps with Alice’s. Recommendation systems, search ranking, and the entire attention mechanism inside a transformer are, at heart, dot products between vectors. The @ symbol is NumPy’s “matrix multiply”; for two 1-D vectors it computes exactly this dot product.

2. Matrix · vector → a vector (apply a transform)

Multiply a (d_out, d_in) matrix W by a (d_in,) vector x and you get a new (d_out,) vector. You have transformed the sample from one space into another — the defining job of a linear layer.

Before you read the output, predict its shape. The input has 4 features and W maps 4 → 2, so how long should y be?

import numpy as np

# One sample: 4 features
x = np.array([1.0, 2.0, 3.0, 4.0])

# A linear layer mapping 4 features -> 2 outputs
W = np.array([
    [0.5, -0.2, 0.1,  0.3],
    [0.0,  0.4, 0.2, -0.1],
])
b = np.array([0.1, -0.1])

y = W @ x + b            # y = Wx + b
print("Input shape:", x.shape, "Output shape:", y.shape)
print("y:", y)

Input shape: (4,) Output shape: (2,)
y: [1.7 0.9]

Four numbers went in, two came out — that is one layer of a neural network. Stack a few of these with a nonlinearity wedged between them and you have a feed-forward network.

3. Matrix · matrix → a matrix (a whole batch at once)

Stack your samples row by row into X of shape (n, d), multiply by W.T of shape (d, d_out), and every sample is transformed in a single step:

import numpy as np

# A batch of 3 samples, 4 features each
X = np.array([
    [1.0, 2.0, 3.0, 4.0],
    [0.5, 1.5, 2.5, 3.5],
    [2.0, 0.0, 1.0, 1.0],
])
W = np.array([
    [0.5, -0.2, 0.1,  0.3],
    [0.0,  0.4, 0.2, -0.1],
])
b = np.array([0.1, -0.1])

Y = X @ W.T + b          # one row of predictions per sample
print("X:", X.shape, "W.T:", W.T.shape, "Y:", Y.shape)
print(Y)

X: (3, 4) W.T: (4, 2) Y: (3, 2)
[[1.7  0.9 ]
 [1.35 0.65]
 [1.5  0.  ]]

Notice the first row of Y — [1.7, 0.9] — is exactly the single-sample answer from before; the matrix simply did it for all three rows together. One matrix multiply has replaced a Python loop over the samples, and with a real batch of ten thousand rows that is the difference between a millisecond and ten seconds. Matrix products also compose: applying W1 then W2 is the same as applying the single product W2 @ W1, so a deep stack of transforms can be reasoned about one multiplication at a time.

4. Broadcasting → per-axis operations without a loop

The quiet glue between the big multiplies is broadcasting: applying a small vector across a whole axis of a matrix, with no loop written by hand.

import numpy as np

X = np.array([
    [10.0, 200.0, 0.5],
    [12.0, 210.0, 0.7],
    [11.0, 205.0, 0.6],
])

# Subtract each column's mean. mean has shape (3,) and broadcasts down the rows
X_centered = X - X.mean(axis=0)
print(X_centered.round(2))

[[-1.  -5.  -0.1]
 [ 1.   5.   0.1]
 [ 0.   0.   0. ]]

One line centred every column on zero. Centring data, scaling features, adding a bias vector to a batch — all broadcasting. It rarely gets named in a paper, yet it is doing work on nearly every line.

In one breath

A dataset is a matrix and a sample is a vector; four operations act on them — the dot product turns two vectors into a similarity number, a matrix-times-vector transforms one sample, a matrix-times-matrix transforms an entire batch (and composes transforms), and broadcasting applies a small vector across an axis without a loop. Learn to spot those four shapes and most ML code reads as data moving, not symbols piling up.

Practice

A question to carry forward

We have leaned on the word vector as if it were obvious — a row of the matrix, a list of d numbers — and we measured how aligned two of them were with a dot product, calling a large value “pointing the same way.” But that phrase quietly smuggled in geometry: pointing, direction, length, angle. A vector, it turns out, is not just a list of numbers; it is an arrow with a length and a heading, and the dot product is really a statement about the angle between two arrows.

So here is the thread onward: what is a vector, really — when is it a point in space and when is it a direction, what does its length mean, and why will “how long is this vector” and “what is the angle between these two” turn out to be the two questions almost every ML method is secretly asking?