Vectors, Matrices & Special Forms

Two objects do most of the heavy lifting in this whole block: a column of numbers and a grid of numbers. The column is a vector — picture an arrow in space. The grid is a matrix — picture a row of arrows stacked side by side. Once adding and multiplying these feels automatic, the rest of Linear Algebra (subspaces, eigenvalues, least squares) stops being mysterious and starts being mechanical. And the exam leans on this block hard, so it’s worth the fluency. Every neural-net forward pass, every batch of data fed through a model, is one of these matrix products under the hood — so this is the literal arithmetic of modern ML.

Adding and scaling — the easy operations

Addition is entry by entry, and it only works when the shapes match. Scalar multiplication just multiplies every entry by the same number. For a vector v = (2, 3), the scaled vector 2v = (4, 6) — twice as long, same direction.

A matrix with m rows and n columns is called an m × n matrix; the entry in row i, column j is written a_ij. Two matrices add only if they are the same size.

Matrix multiplication — row dot column

This is the operation that trips people up, so slow down. To multiply A (size m × k) by B (size k × n), the inner dimensions must match (k = k). Entry (i, j) of the product is the dot product of row i of A with column j of B: multiply matching elements and sum them.

Step through the full row-by-column animation for a larger product here — watch the active row and column light up as each output cell fills:

The shape rule in one line: (m × k) times (k × n) gives (m × n). The two ks in the middle must agree and they vanish; the outer dimensions survive.

The transpose — flip across the diagonal

The transpose Aᵀ turns rows into columns: entry (i, j) becomes entry (j, i). A 2 × 3 matrix becomes 3 × 2. The one rule GATE tests is that the transpose of a product reverses the order:

(AB)ᵀ = Bᵀ Aᵀ      (order flips — not Aᵀ Bᵀ)

How GATE asks this

These ideas are rarely a question on their own — they are the machinery inside nearly every Linear Algebra problem. Directly, GATE DA poses short MCQ/NAT items: compute a specific entry of a product AB, decide whether AB = BA for a given pair, or pick the correct simplification of (AB)ᵀ. The recurring decision is always the same: line up the inner dimensions, then dot row with column.

Worked example — a product, both ways

Let A = [[1, 2], [0, 1]] and B = [[1, 0], [3, 1]]. Compute AB and BA.

Apply row-dot-column for each entry of AB:

A = [1 2]      B = [1 0]
    [0 1]          [3 1]

AB[1,1] = 1·1 + 2·3 = 7      AB[1,2] = 1·0 + 2·1 = 2
AB[2,1] = 0·1 + 1·3 = 3      AB[2,2] = 0·0 + 1·1 = 1

AB = [7 2]
     [3 1]

Now reverse the order and recompute:

BA[1,1] = 1·1 + 0·0 = 1      BA[1,2] = 1·2 + 0·1 = 2
BA[2,1] = 3·1 + 1·0 = 3      BA[2,2] = 3·2 + 1·1 = 7

BA = [1 2]
     [3 7]

AB ≠ BA — same two matrices, different products. Order is part of the operation.

Special matrices — the named shapes

GATE expects you to recognise these on sight:

• Identity I — 1s on the main diagonal, 0s elsewhere. It is the “do nothing” matrix: AI = IA = A.
• Diagonal — nonzero entries only on the main diagonal. Multiplying by one just scales each row (or column).
• Symmetric — equal to its own transpose: A = Aᵀ. The entry a_ij equals a_ji, so it mirrors across the main diagonal. Covariance matrices are symmetric.
• Upper / lower triangular — all entries below (resp. above) the main diagonal are zero. Their determinant is just the product of the diagonal entries.

Quick check