Vectors, Matrices & Special Forms

The probability roadmap is behind us; this one is built from just two objects.

A column of numbers is an arrow pointing somewhere in space. A grid of numbers is a row of those arrows stacked side by side.

Once adding and multiplying these two feels automatic, the rest of Linear Algebra — subspaces, eigenvalues, least squares — stops being mysterious and turns mechanical. The exam leans on this block hard, so the fluency is worth it.

The easy operations: add and scale

The column is a vector, the grid a matrix (an m × n matrix has m rows and n columns; the entry in row i, column j is a_ij). Both add entry by entry, and only when the shapes match: (2,3) + (5,1) = (7,4). Scaling just multiplies every entry by the same number — for v = (2, 3), the scaled 2v = (4, 6) is twice as long in the same direction. Nothing surprising yet.

Matrix multiplication — row dot column

This is the operation that trips people up, so slow down. To multiply A (size m × k) by B (size k × n), the inner dimensions must match. Entry (i, j) of the product is the dot product of row i of A with column j of B: pair up the matching numbers, multiply, and sum.

The shape rule in a line: (m × k) times (k × n) gives (m × n) — the two ks in the middle must agree and they vanish, while the outer dimensions survive. Step through a larger product below, watching the active row and column light up as each output cell fills:

Order matters, and the transpose reverses it

Here is the habit that breaks intuition from ordinary numbers: matrix multiplication is not commutative — AB is generally not BA. Take A = [[1, 2], [0, 1]] and B = [[1, 0], [3, 1]], and compute both:

AB:  (1·1+2·3, 1·0+2·1)   = [7  2]        BA:  (1·1+0·0, 1·2+0·1)   = [1  2]
     (0·1+1·3, 0·0+1·1)     [3  1]              (3·1+1·0, 3·2+1·1)     [3  7]

Same two matrices, different products — AB ≠ BA. Order is part of the operation.

The precise meaning of “not commutative”

Read that claim precisely, though, because people routinely over-correct it. “Not commutative” means AB and BA need not be equal — not that they are always different. Plenty of pairs do commute:

• any matrix with the identity (AI = IA = A);
• any matrix with itself;
• any two diagonal matrices of the same size.

The shapes need not even line up for the question to make sense. If A is 2 × 3 and B is 3 × 2, then AB is 2 × 2 while BA is 3 × 3, so “are they equal?” cannot be asked at all. The rule is that you may never assume AB = BA; a particular pair may still oblige.

The transpose Aᵀ flips a matrix across its diagonal, turning rows into columns, so a 2 × 3 becomes 3 × 2. The one transpose fact GATE keeps testing is that the transpose of a product reverses the order:

(AB)ᵀ = Bᵀ Aᵀ        (the order flips — NOT Aᵀ Bᵀ)

The named shapes

A few special matrices you must recognise on sight:

• Identity I — 1s on the main diagonal, 0s elsewhere; the “do nothing” matrix, AI = IA = A.
• Diagonal — non-zero only on the main diagonal; multiplying by it just scales each row (or column).
• Symmetric — equal to its own transpose, A = Aᵀ, so a_ij = a_ji (it mirrors across the diagonal). Covariance matrices are symmetric.
• Upper / lower triangular — zeros below (resp. above) the diagonal. Their determinant is just the product of the diagonal entries. Note that a triangular matrix is generally not symmetric: a_ij and a_ji sit on opposite sides of the diagonal, and only one of those sides is forced to zero. A diagonal matrix, by contrast, is always symmetric — both sides are zero.

A question to carry forward

A matrix takes a vector and sends it to a new vector — Av lands somewhere.

Here is the thread onward. If you take *all the vectors you can reach by adding and scaling a handful of arrows, what shape does that collection make in space? Which such collections are “complete” enough to count as a little world of their own?

In one breath

• Add/scale entry-by-entry (shapes must match); an m×n matrix has entry a_ij in row i, col j.
• Multiply row-dot-column: (m×k)·(k×n) = (m×n) — inner dims match and vanish. AB ≠ BA in general (not commutative).
• Transpose flips across the diagonal; the product rule reverses order: (AB)ᵀ = BᵀAᵀ (not AᵀBᵀ).
• Named shapes: identity I (AI=IA=A), diagonal (scales rows/cols), symmetric (A=Aᵀ, e.g. covariance), triangular (det = product of the diagonal).
• GATE distractors: the wrong-order (AB)ᵀ = AᵀBᵀ and assuming AB = BA — check the order every time.

Practice