Independence, Span, Basis & Dimension
Linear independence, span, basis and dimension in one pass — plus why orthonormal sets are automatically independent and why a space has many valid bases, not one.
What you'll learn
- Linear independence: the only solution to the sum of c_i v_i = 0 is all c_i = 0
- Span, basis (an independent spanning set), and dimension (the count of basis vectors)
- Orthonormal sets are mutually perpendicular unit vectors and are automatically independent
- A vector space has MANY valid bases — independence is weaker than orthogonality
Before you start
Pick a handful of arrows. How small can you make that handful and still reach every point in the space using only additions and scalings of them? That’s the question this lesson lives on. Four tightly linked words come out of it — independence, span, basis, dimension — and GATE tests them as a cluster. The mental picture is simple: independent vectors each point in a genuinely new direction the others can’t reach, and a basis is just enough of those to build the whole space without anything redundant. This is the exact language data scientists use when two features are perfectly correlated (one is a multiple of another — redundant, “dependent”) or when they ask how many truly independent directions a dataset actually has.
Linear independence — no redundant directions
Vectors v₁, …, vₙ are linearly independent when no one of them is a
combination of the others. The crisp algebraic test: the only way to make
c₁v₁ + c₂v₂ + … + cₙvₙ = 0is to take all coefficients c_i = 0. If some nonzero choice of coefficients also
gives zero, the vectors are dependent — at least one is redundant.
Drag the two arrows. When they point in different directions, every combination of them fills out the plane — independence in action. Now drag one to lie on top of (or opposite) the other and the combinations collapse onto a single line: the hallmark of dependence.
Span, basis, dimension
The span of a set of vectors is everything you can build from them by addition and
scaling — all their linear combinations. Two independent vectors in R² span the
whole plane; one nonzero vector spans only a line.
A basis of a space is a set that is both linearly independent and spans the
space — enough vectors to reach everything, with none redundant. The dimension
is simply the number of vectors in a basis (every basis of a given space has the
same count). R² has dimension 2, R³ has dimension 3.
A space has many valid bases — any set of n independent vectors that spans an
n-dimensional space qualifies. There is nothing unique about the “standard” one.
Orthonormal sets — perpendicular and unit length
A set is orthonormal when the vectors are mutually perpendicular (every pair
has dot product 0) and each has length 1. The standard basis (1,0) and
(0,1) is orthonormal. A key fact GATE tests: an orthonormal set is automatically
linearly independent — perpendicular directions can never be combinations of one
another. But the converse is weaker: independent vectors need not be orthogonal.
How GATE asks this
GATE DA poses these as MCQ/MSQ items — and it appeared in 2025. Typical
prompts: “are these vectors linearly independent?”, “which of these sets form a basis
of R³?”, or a multi-select on properties of orthonormal sets. The 2025 trap played
on uniqueness — assuming a space has a single orthonormal basis. Remember: many
orthonormal bases exist (rotate the standard one and it is still orthonormal). To check
independence fast for n vectors in Rⁿ, form a matrix and ask whether its
determinant is nonzero (nonzero means independent).
Worked example — independent vs dependent
(1) Are
(1, 0)and(1, 1)independent, and do they form a basis ofR²? (2) Are(1, 2)and(2, 4)independent?
Pair 1 — independent, and a basis. Solve c₁(1,0) + c₂(1,1) = (0,0). The second
coordinate gives c₂ = 0; the first then gives c₁ + 0 = 0, so c₁ = 0. Only the
all-zero solution exists, so they are independent. Two independent vectors in the
2-dimensional space R² automatically span it, so they form a basis — note this is
a different basis from the standard (1,0), (0,1).
Pair 2 — dependent. Observe (2, 4) = 2·(1, 2): the second vector is exactly twice
the first. So 2·(1,2) − 1·(2,4) = (0,0) is a nonzero combination giving zero. They
are dependent and span only a line, not the plane.
det [1 1] = 1·1 − 0·1 = 1 ≠ 0 → independent (a basis of R²)
[0 1]
det [1 2] = 1·4 − 2·2 = 0 → dependent
[2 4]Quick check
Quick check
Practice this in an interview
All questionsOLS linear regression rests on five assumptions: linearity, independence of errors, homoscedasticity, normality of residuals, and no perfect multicollinearity. Violating any one of them degrades coefficient estimates, standard errors, or the validity of hypothesis tests.
PCA finds the orthogonal directions of maximum variance in the data and projects onto a lower-dimensional subspace, reducing features while retaining most information. It is most useful before distance-based models or when training is bottlenecked by dimensionality. Its main limits are loss of interpretability, sensitivity to scale, and an assumption of linear structure.
As the number of features grows, the volume of the feature space increases exponentially, so training data becomes exponentially sparse. Distance-based algorithms degrade because points become approximately equidistant; density estimation requires data that grows exponentially; and overfitting risk rises for any fixed training set size.
1NF eliminates repeating groups and requires atomic column values. 2NF further removes partial dependencies on a composite key. 3NF removes transitive dependencies — every non-key column must depend on the key, the whole key, and nothing but the key. Denormalization trades update anomalies for read performance, and is appropriate when the read path dominates and write correctness can be enforced at the application layer or with materialized views.