5.1 The Scalar Product in ℝⁿ · Linear Algebra Refresher

Adding a scalar product to

R^{n}

gives you length, angle, and — most importantly — projection. Projection is the engine behind least squares, Gram–Schmidt, and the whole geometric reading of the four subspaces.

Definition · Scalar product, length, distance

For

x, y \in R^{n}

, the scalar (dot) product is

x^{T} y = x_{1} y_{1} + \dots + x_{n} y_{n}

. The Euclidean length is

∥ x ∥ = (x^{T} x)^{1/2}

, and the distance between

x

and

y

∥ x - y ∥

The law of cosines applied to the triangle with sides $x$ , $y$ , $x - y$ gives the angle formula:

x^{T} y = ∥ x ∥ ∥ y ∥ cos θ, 0 \leq θ \leq π .

Cauchy–Schwarz

∣ x^{T} y ∣ \leq ∥ x ∥ ∥ y ∥

, with equality iff

x

and

y

are linearly dependent.

It's what makes $cos θ$ a legitimate cosine — the quotient never leaves $[- 1, 1]$ .

Definition · Orthogonality

x ⊥ y

means

x^{T} y = 0

, i.e.

θ = π /2

(or one of them is

0

Projection — the workhorse

α = \frac{x ^{T} y}{∥ y ∥} (scalar projection), p = α \frac{y}{∥ y ∥} = \frac{x ^{T} y}{y ^{T} y} y (vector projection) .

★Key idea

Two checks that catch every projection error: (1)

p

must be a multiple of the vector you projected onto; (2)

x - p

must be orthogonal to that vector. Verify (2) in one dot product and you're safe.

Worked example

Angle and projection in $R^{3}$

0/4 steps

For

x = (1, 2, 2)^{T}

and

y = (3, 0, 4)^{T}

, find the angle between them and the vector projection of

x

onto

y

Pythagorean law

x^{T} y = 0

then

∥ x + y ∥^{2} = ∥ x ∥^{2} + ∥ y ∥^{2}

Expand $∥ x + y ∥^{2} = (x + y)^{T} (x + y)$ and the cross terms vanish. §5.4 shows this needs a norm that comes from an inner product.

Check yourself

The vector projection of

x

onto

y

is:

T / F

The zero vector is orthogonal to every vector in

R^{n}

T / F

Equality holds in

∣ x^{T} y ∣ \leq ∥ x ∥∥ y ∥

if and only if

x

and

y

are linearly dependent.

5.1The Scalar Product in ℝⁿ

What you must be able to do

Projection — the workhorse

Angle and projection in $R^{3}$

Check yourself

5.1The Scalar Product in ℝⁿ

What you must be able to do

Projection — the workhorse

Angle and projection in R3

Check yourself

Angle and projection in $R^{3}$