5.5 Orthonormal Sets · Linear Algebra Refresher

An orthonormal basis is the best coordinate system money can buy: coordinates are single inner products, no linear system to solve, and lengths behave. Orthogonal matrices are the square case.

Definition · Orthogonal and orthonormal sets

{v_{1}, \dots, v_{n}}

is orthogonal if

⟨ v_{i}, v_{j} ⟩ = 0

whenever

i \neq = j

. It is orthonormal if additionally

∥ v_{i} ∥ = 1

for every

i

. Any orthogonal set of nonzero vectors becomes orthonormal after dividing each vector by its length.

Theorem 5.5.1

An orthogonal set of nonzero vectors is linearly independent.

Take the inner product of $\sum c_{i} v_{i} = 0$ with $v_{j}$ : all cross terms die, leaving $c_{j} ∥ v_{j} ∥^{2} = 0$ , so $c_{j} = 0$ .

Theorem 5.5.2 & Corollary 5.5.4 · Coordinates for free

{u_{1}, \dots, u_{n}}

is an orthonormal basis, then for every

v

v = i = 1 \sum n c_{i} u_{i} with c_{i} = ⟨ v, u_{i} ⟩, ∥ v ∥^{2} = i = 1 \sum n c_{i}^{2} (Parseval) .

Compare §3.5, where finding coordinates meant inverting a matrix. Orthonormality replaces $U^{- 1}$ with $U^{T}$ .

Worked example

Coordinates and Parseval

0/4 steps

Verify that

u_{1} = \frac{1}{3} (2, 2, 1)^{T}

u_{2} = \frac{1}{3} (2, - 1, - 2)^{T}

u_{3} = \frac{1}{3} (1, - 2, 2)^{T}

is an orthonormal basis of

R^{3}

, then expand

x = (1, 2, 3)^{T}

in it.

Orthogonal matrices

Definition · Orthogonal matrix

n \times n

matrix

Q

is orthogonal if its column vectors form an orthonormal set, equivalently

Q^{T} Q = I

, equivalently

Q^{- 1} = Q^{T}

. Then

Q Q^{T} = I

too, so the rows are orthonormal as well.

!Exam trap

An 'orthogonal matrix' needs orthonormal columns, not merely orthogonal ones.

(2003)

has orthogonal columns but

Q^{T} Q \neq = I

. Unfortunate naming — blame history.

$⟨ Q x, Q y ⟩ = x^{T} Q^{T} Q y = x^{T} y$ — angles preserved.
$∥ Q x ∥_{2} = ∥ x ∥_{2}$ — lengths preserved. Rotations and reflections are exactly the orthogonal matrices.
$det (Q) = \pm 1$ , since $det (Q)^{2} = det (Q^{T} Q) = 1$ . ( $+ 1$ rotation, $- 1$ reflection.)
Products of orthogonal matrices are orthogonal.

Theorem 5.5.6 & Corollary 5.5.9 · Least squares gets easy

If the columns of

U

are orthonormal, then the least squares solution of

U x = b

x^= U^{T} b

and the projection onto

R (U)

p = U U^{T} b

Compare $A (A^{T} A)^{- 1} A^{T}$ from §5.3: orthonormality collapses $(A^{T} A)^{- 1}$ to $I$ . This is the entire motivation for Gram–Schmidt in §5.6.

Check yourself

Q

is an

n \times n

orthogonal matrix. Which statement is false?

T / F

Any orthogonal set of nonzero vectors is linearly independent.

{u_{1}, \dots, u_{k}}

is an orthonormal basis for a subspace

S \subseteq R^{m}

, and

U = (u_{1}, \dots, u_{k})

. The projection of

b

onto

S

is:

5.5Orthonormal Sets

What you must be able to do

Coordinates and Parseval

Orthogonal matrices

Check yourself