The big picture

Most exam difficulty comes from not recognising that two questions are the same question. This page collects the places where the course loops back on itself — the equivalence chain, the rank table, and the three ideas that everything else is built from.

The equivalence chain

For a square $n \times n$ matrix $A$ , all of the following say the same thing. If you prove one, you have them all — and exam questions constantly ask you to jump between distant entries on this list.

Ch 1

$A$ is nonsingular — $A^{- 1}$ exists
$A x = 0$ has only the trivial solution
$A$ is row equivalent to $I$
$A$ is a product of elementary matrices
$A x = b$ has a unique solution for every $b$

Ch 2

$det (A) \neq = 0$

Ch 3

The columns of $A$ are linearly independent
The columns of $A$ form a basis for $R^{n}$
The rows of $A$ are linearly independent
$rank (A) = n$
$N (A) = {0}$ , i.e. $dim N (A) = 0$
$R (A) = R^{n}$ — the column space is everything

Ch 4

$L_{A} (x) = A x$ is one-to-one and onto
$ker (L_{A}) = {0}$

Ch 6

$0$ is not an eigenvalue of $A$

Only for square matrices

Every line above assumes

A

n \times n

. For a rectangular

A

, 'nonsingular' and '

det

' are undefined, and independence of the columns no longer implies independence of the rows. Rectangular questions go through the rank table below instead.

The rank decision table

This is the rectangular replacement for the equivalence chain, and it answers the whole family of 'rank 5, six rows — how many solutions?' questions. You always need both $m$ and $n$ ; rank alone is never enough.

$A$ is $m \times n$ , $rank = r$	Existence	Uniqueness	Solution count
$r = m = n$	always	unique	exactly 1 for every $b$
$r = n < m$ (tall, full column rank)	only if $b \in R (A)$	unique	0 or 1 — the least squares setting
$r = m < n$ (wide, full row rank)	always	never unique	$\infty$ , with $n - r$ parameters
$r < m$ and $r < n$	only if $b \in R (A)$	never unique	0 or $\infty$

Compare

r

with

m

for existence, and

r

with

n

for uniqueness. That's the whole table.

A

6 \times 8

rank = 5

. Consistent system — how many solutions?

dim N (A) = 8 - 5 = 3

, so infinitely many, with 3 free parameters. Never exactly one, because

r < n

A

6 \times 5

rank = 5

. How many solutions?

r = n

so the null space is trivial: 0 or 1. Since

r < m

, the column space is a proper subspace of

R^{6}

, so most

b

give none.

A

4 \times 7

. Can

A x = b

ever have a unique solution?

No.

r \leq min (4, 7) = 4 < 7 = n

, so there is always at least one free variable. Underdetermined systems have 0 or

\infty

solutions.

A

5 \times 5

with

dim N (A) = 2

. Is

det (A) = 0

Yes. Nullity 2 means

rank = 3 < 5

, so

A

is singular — every entry of the equivalence chain fails at once.

One question asked six ways

Chapter	Same question, new vocabulary
§1.2 Echelon form	Is there a row $(0 \dots 0 ∣ 1)$ ? If not, consistent.
§1.3 Columns	Is $b$ a linear combination of the columns of $A$ ?
§2.2 Determinant	For square $A$ : $det (A) \neq = 0$ ⟹ consistent for every $b$ , uniquely.
§3.6 Column space	Is $b \in R (A)$ ? And $dim R (A) = rank (A)$ .
§4.1 Range	Is $b$ in the range of the transformation $L_{A}$ ?
§5.2 Orthogonality	Is $b ⊥ N (A^{T})$ ? (Because $R (A) = N (A^{T})^{⊥}$ .)
§5.3 When it's not	Give up on equality — project $b$ onto $R (A)$ and solve $A^{T} A x^= A^{T} b$ .

Six chapters, one question: is $b$ reachable from the columns of $A$ ?

Three ideas the whole course runs on

Elimination

Row reduce and read the pivots. It solves systems (§1.2), inverts matrices (§1.5), computes determinants (§2.2), tests independence (§3.3), produces bases for all three subspaces (§3.6), builds matrix representations (§4.2), finds null spaces of $A - λ I$ (§6.1), and solves the normal equations (§5.3). If you're stuck, row reduce something.

Projection

Decompose a vector into 'the part inside the subspace' plus 'the part orthogonal to it'. That's the vector projection (§5.1), the direct sum $R^{n} = S \oplus S^{⊥}$ (§5.2), the least squares residual condition (§5.3), and each Gram–Schmidt step (§5.6). The residual being orthogonal is what makes the answer optimal.

Choosing a basis

The same object looks different in different coordinates. A vector becomes a coordinate list (§3.5); a transformation becomes a matrix (§4.2); changing basis conjugates it to $S^{- 1} A S$ (§6.1). Diagonalization (§6.3) is the punchline: pick the eigenvector basis and the matrix becomes diagonal, so powers and exponentials become scalar arithmetic.

Triage: what do I reach for?

Reading an exam problem

?
'Solve / is it consistent / how many solutions'
Row reduce $(A ∣ b)$ , then use the rank table. §1.2, §3.6
?
'Singular / invertible / does $A^{- 1}$ exist'
Determinant if it's small, otherwise row reduce and count pivots. Then quote the equivalence chain. §1.5, §2.2
?
'Independent / basis / dimension / span'
Stack the vectors as columns and row reduce; pivots count the independent ones. Square ⟹ determinant shortcut. §3.3, §3.4, §3.6
?
'Closest / best fit / minimise the error'
Least squares. Build $A$ from the model and solve $A^{T} A x^= A^{T} b$ . §5.3
?
'Orthonormal / orthogonal basis / QR'
Gram–Schmidt, subtracting projections at each step. §5.5, §5.6
?
' $λ$ / diagonalize / $A^{k}$ / $Y^{'} = A Y$ / $e^{A}$ '
Characteristic polynomial, then null space of $A - λ I$ for each root. §6.1–§6.3

When the wording is unfamiliar, translate it into one of these six buckets first.

The formulas worth memorising cold

Projection onto a vector

p = \frac{x ^{T} y}{y ^{T} y} y

Normal equations

A^{T} A x^= A^{T} b

Projection matrix

P = A (A^{T} A)^{- 1} A^{T}, P^{2} = P = P^{T}

Rank–Nullity

rank (A) + dim N (A) = n

Complement dimensions

dim S + dim S^{⊥} = n, R^{n} = S \oplus S^{⊥}

Fundamental subspaces

N (A) = R (A^{T})^{⊥}, N (A^{T}) = R (A)^{⊥}

Characteristic polynomial ($2\times2$)

p (λ) = λ^{2} - tr (A) λ + det (A)

Trace and determinant

tr (A) = \sum λ_{i}, det (A) = \prod λ_{i}

Diagonalization

A = X D X^{- 1} \Rightarrow A^{k} = X D^{k} X^{- 1}, e^{A} = X e^{D} X^{- 1}

Linear ODE system

Y^{'} = A Y \Rightarrow Y = \sum c_{i} e^{λ_{i} t} x_{i}

Order reversal

(A B)^{- 1} = B^{- 1} A^{- 1}, (A B)^{T} = B^{T} A^{T}

Determinant scaling

det (α A) = α^{n} det (A), det (A B) = det A det B

Test yourself →Back to §1.1