Projections & Idempotent Matrices

The last lesson’s matrix preserved everything, turning space rigidly. Here is its opposite — a matrix that throws information away on purpose. Shine a light straight down on a vector and look at the shadow it casts on the floor. That shadow is a projection — the vector squashed onto a lower-dimensional subspace. And shadows have a quirk that is obvious once you say it aloud: the shadow of a shadow is the same shadow. A point already on the floor projects to itself, so applying the projection a second time changes nothing.

That “twice changes nothing” is exactly the equation P² = P. Matrices with this property are called idempotent, and from that single fact the rank, the nullity, and the allowed eigenvalues all fall out — which is why GATE keeps reaching for it.

Projection onto a subspace

A projection P sends each vector to the closest point in some subspace U, and leaves vectors already living in U untouched. Picture dropping a perpendicular onto a line:

Three properties follow, and they are the whole exam syllabus here:

• Idempotent: P² = P (so P³ = P·P² = P, and every higher power too).
• Eigenvalues in {0, 1}: if Pv = λv then Pv = P²v = λ²v, so λ² = λ, giving λ = 0 or 1. Vectors in U keep their length (eigenvalue 1); vectors perpendicular to U are killed (eigenvalue 0).
• Rank and nullity: the output of P fills exactly U, so rank(P) = dim(U), and rank-nullity gives nullity(P) = n − dim(U) — the nullity being the dimension of the set of vectors that P flattens to zero, here precisely the directions perpendicular to U.

An orthogonal projection (the perpendicular kind, as drawn) carries one extra property: it is symmetric, P = Pᵀ. Not every idempotent matrix is symmetric — oblique projections drop the shadow at a slant — but the perpendicular ones are. Set the playground matrix to [[1, 0], [0, 0]] (projection onto the x-axis): the unit shape collapses to a line segment, points already on the line stay put, and applying it again gives the identical result — P² = P in pictures.

The centering matrix — PCA’s engine

The single most important projection in data analysis subtracts the mean from a data vector. With the all-ones vector 1, the centering matrix is

C = I − (1/n)·11ᵀ

Here 11ᵀ is the n × n all-ones matrix, so (1/n)·11ᵀ averages the entries and Cx = x − mean(x)·1 removes that average. C is symmetric and idempotent — the orthogonal projection onto the mean-zero subspace — and PCA begins by applying C to every feature, which is why this matrix turns up in DA papers.

One notational trap sits right at the heart of that formula, and it catches almost everyone once: 1ᵀ1 and 11ᵀ are not the same object. 1ᵀ1 is a row times a column — a single number, the sum of n ones, so 1ᵀ1 = n. 11ᵀ is a column times a row — an entire n × n matrix with 1 in every entry. Same two symbols, opposite order, scalar versus matrix. Read them the wrong way round and the C² = C computation below turns to nonsense.

Run C on three numbers to see the shadow. Take x = (1, 2, 6), so n = 3 and the mean is 9/3 = 3. Then Cx = (1−3, 2−3, 6−3) = (−2, −1, 3), whose entries sum to zero — it has landed in the mean-zero subspace. Now centre it again: its mean is already 0, so subtracting that mean changes nothing and C(Cx) = Cx. That is C² = C, in three numbers.

A worked example — the 2024 and 2026 questions

Let M be the orthogonal projection of R³ onto a 2-dimensional subspace U. Evaluate M², M³, rank(M), and nullity(M).

A projection satisfies M² = M, so M³ = M·M² = M as well — every positive power equals itself. Its output fills exactly U, so rank(M) = dim(U) = 2; and rank-nullity in R³ gives nullity(M) = 3 − 2 = 1 — the one direction perpendicular to U, the part the shadow discards.

Now the 2026 centering-matrix part. Show C² = C for C = I − (1/n)11ᵀ, using the one key fact 1ᵀ1 = n (summing n ones):

C² = (I − (1/n)11ᵀ)(I − (1/n)11ᵀ)
   = I − (1/n)11ᵀ − (1/n)11ᵀ + (1/n²)·1(1ᵀ1)1ᵀ
   = I − (2/n)11ᵀ + (1/n²)·1·(n)·1ᵀ        [1ᵀ1 = n]
   = I − (2/n)11ᵀ + (1/n)11ᵀ
   = I − (1/n)11ᵀ = C   ✓

The two −(1/n)11ᵀ terms and the recovered +(1/n)11ᵀ collapse back to one, proving C is idempotent.

A question to carry forward

Symmetric matrices keep appearing — orthogonal projections are symmetric, covariance matrices are symmetric. They clearly have a special structure. Here is the thread onward: feed a vector into the expression xᵀA x and you get a single number, a kind of warped “size” of x. For a symmetric A, when is that number always positive, and why would anyone care?

In one breath

• A projection casts a shadow onto a subspace U; doing it twice changes nothing → idempotent P² = P (so Pᵏ = P).
• Eigenvalues ∈ 1 (λ²=λ): 1 for vectors kept (in U), 0 for vectors killed (perpendicular).
• rank(P) = dim(U), nullity(P) = n − dim(U) (rank-nullity).
• Orthogonal projection ⇒ symmetric (P=Pᵀ); oblique ones are idempotent but not symmetric. Only the identity is an invertible projection.
• Centering matrix C = I − (1/n)11ᵀ: symmetric, idempotent (1ᵀ1=n makes C²=C), subtracts the mean — PCA’s first step. rank = n−1.

Practice