5.4 Inner Product Spaces · Linear Algebra Refresher

Abstracting the dot product to arbitrary vector spaces lets you talk about the angle between two polynomials or the distance between two matrices. The payoff is that every projection formula from §5.1 transfers verbatim.

Definition · Inner product space

An inner product on a real vector space

V

assigns to each pair

u, v

a scalar

⟨ u, v ⟩

satisfying: (I)

⟨ v, v ⟩ \geq 0

, with equality iff

v = 0

; (II)

⟨ u, v ⟩ = ⟨ v, u ⟩

; (III)

⟨ α u + β w, v ⟩ = α ⟨ u, v ⟩ + β ⟨ w, v ⟩

Space	Inner product	Induced norm
$R^{n}$	$x^{T} y$ , or weighted $\sum x_{i} y_{i} w_{i}$ with $w_{i} > 0$	$∥ x ∥_{2}$
$R^{m \times n}$	$⟨ A, B ⟩ = \sum_{i} \sum_{j} a_{ij} b_{ij}$	Frobenius $∥ A ∥_{F}$
$C [a, b]$	$\int_{a}^{b} f (x) g (x) d x$ , or with weight $w (x) > 0$	$(\int f^{2})^{1/2}$
$P_{n}$	$\sum_{i = 1}^{n} p (x_{i}) q (x_{i})$ at $n$ distinct points	$(\sum p (x_{i})^{2})^{1/2}$

!Exam trap

Weights must be strictly positive. A single negative weight breaks axiom (I) — you could get

⟨ v, v ⟩ < 0

for a nonzero

v

, and then

∥ v ∥

isn't even real.

All the §5.1 machinery transfers with $x^{T} y$ replaced by $⟨ x, y ⟩$ : orthogonality, the Pythagorean law $∥ u + v ∥^{2} = ∥ u ∥^{2} + ∥ v ∥^{2}$ , Cauchy–Schwarz $∣ ⟨ u, v ⟩ ∣ \leq ∥ u ∥∥ v ∥$ , and the projections

α = \frac{⟨ u , v ⟩}{∥ v ∥}, p = \frac{⟨ u , v ⟩}{⟨ v , v ⟩} v .

Worked example

The angle between two functions

0/3 steps

C [0, 1]

with

⟨ f, g ⟩ = \int_{0}^{1} f (x) g (x) d x

, find the angle between

f (x) = 1

and

g (x) = x

Norms without inner products

Definition · Normed linear space

A norm satisfies

∥ v ∥ \geq 0

with equality iff

v = 0

;

∥ α v ∥ = ∣ α ∣ ∥ v ∥

; and the triangle inequality

∥ u + v ∥ \leq ∥ u ∥ + ∥ v ∥

. The

p

-norms on

R^{n}

are

∥ x ∥_{p} = (i = 1 \sum n ∣ x_{i} ∣^{p})^{1/ p}, ∥ x ∥_{\infty} = i max ∣ x_{i} ∣.

!Pythagoras is not universal

Only

∥ \cdot ∥_{2}

comes from an inner product. Take the orthogonal pair

x_{1} = (1, 2)^{T}

x_{2} = (- 4, 2)^{T}

: in the

\infty

-norm,

∥ x_{1} ∥_{\infty}^{2} + ∥ x_{2} ∥_{\infty}^{2} = 4 + 16 = 20

but

∥ x_{1} + x_{2} ∥_{\infty}^{2} = ∥ (- 3, 4)^{T} ∥_{\infty}^{2} = 16

. Orthogonality itself only makes sense when there's an inner product.

Check yourself

Which of these fails to be an inner product on the given space?

T / F

x

and

y

are orthogonal, then

∥ x + y ∥_{\infty}^{2} = ∥ x ∥_{\infty}^{2} + ∥ y ∥_{\infty}^{2}

T / F

Every inner product on

V

induces a norm via

∥ v ∥ = ⟨ v, v ⟩

, but not every norm comes from an inner product.

5.4Inner Product Spaces

What you must be able to do

The angle between two functions

Norms without inner products

Check yourself