Vectors, Matrices & Special Forms

The probability roadmap is behind us; this one is built from just two objects. A column of numbers — picture an arrow pointing somewhere in space. And a grid of numbers — picture a row of those arrows stacked side by side. Once adding and multiplying these two feels automatic, the rest of Linear Algebra — subspaces, eigenvalues, least squares — stops being mysterious and turns mechanical. The exam leans on this block hard, so the fluency is worth it.

The easy operations: add and scale

The column is a vector, the grid a matrix (an m × n matrix has m rows and n columns; the entry in row i, column j is a_ij). Both add entry by entry, and only when the shapes match: (2,3) + (5,1) = (7,4). Scaling just multiplies every entry by the same number — for v = (2, 3), the scaled 2v = (4, 6) is twice as long in the same direction. Nothing surprising yet.

Matrix multiplication — row dot column

This is the operation that trips people up, so slow down. To multiply A (size m × k) by B (size k × n), the inner dimensions must match. Entry (i, j) of the product is the dot product of row i of A with column j of B: pair up the matching numbers, multiply, and sum.

The shape rule in a line: (m × k) times (k × n) gives (m × n) — the two ks in the middle must agree and they vanish, while the outer dimensions survive. Step through a larger product below, watching the active row and column light up as each output cell fills:

Order matters, and the transpose reverses it

Here is the habit that breaks intuition from ordinary numbers: matrix multiplication is not commutative — AB is generally not BA. Take A = [[1, 2], [0, 1]] and B = [[1, 0], [3, 1]], and compute both:

AB:  (1·1+2·3, 1·0+2·1)   = [7  2]        BA:  (1·1+0·3, 1·2+0·1)   = [1  2]
     (0·1+1·3, 0·0+1·1)     [3  1]              (3·1+1·0, 3·2+1·1)     [3  7]

Same two matrices, different products — AB ≠ BA. Order is part of the operation.

The transpose Aᵀ flips a matrix across its diagonal, turning rows into columns, so a 2 × 3 becomes 3 × 2. The one transpose fact GATE keeps testing is that the transpose of a product reverses the order:

(AB)ᵀ = Bᵀ Aᵀ        (the order flips — NOT Aᵀ Bᵀ)

The named shapes

A few special matrices you must recognise on sight:

• Identity I — 1s on the main diagonal, 0s elsewhere; the “do nothing” matrix, AI = IA = A.
• Diagonal — non-zero only on the main diagonal; multiplying by it just scales each row (or column).
• Symmetric — equal to its own transpose, A = Aᵀ, so a_ij = a_ji (it mirrors across the diagonal). Covariance matrices are symmetric.
• Upper / lower triangular — zeros below (resp. above) the diagonal; their determinant is just the product of the diagonal entries.

A question to carry forward

A matrix takes a vector and sends it to a new vector — Av lands somewhere. Here is the thread onward: if you take all the vectors you can reach by adding and scaling a handful of arrows, what shape does that collection make in space, and which such collections are “complete” enough to count as a little world of their own?

In one breath

• Add/scale entry-by-entry (shapes must match); an m×n matrix has entry a_ij in row i, col j.
• Multiply row-dot-column: (m×k)·(k×n) = (m×n) — inner dims match and vanish. AB ≠ BA in general (not commutative).
• Transpose flips across the diagonal; the product rule reverses order: (AB)ᵀ = BᵀAᵀ (not AᵀBᵀ).
• Named shapes: identity I (AI=IA=A), diagonal (scales rows/cols), symmetric (A=Aᵀ, e.g. covariance), triangular (det = product of the diagonal).
• GATE distractors: the wrong-order (AB)ᵀ = AᵀBᵀ and assuming AB = BA — check the order every time.

Practice