1.3 Matrix Arithmetic · Linear Algebra Refresher

The single most important reframing in the course:

A x

is a linear combination of the columns of $A$ . Once you see that, consistency, column space, rank, and least squares all become the same question asked in different accents.

Notation: $a_{ij}$ is the entry in row $i$ , column $j$ . Column vectors are boldface lowercase, $a_{j}$ is the $j$ th column of $A$ , and $a_{i}$ (with the arrow) is the $i$ th row. An $n \times 1$ matrix is a vector in $R^{n}$ .

Two ways to see $A x$

Row picture

The

i

th entry of

A x

is the scalar product

a_{i} x = a_{i 1} x_{1} + \dots + a_{in} x_{n}

. This is how you compute.

Column picture

A x = x_{1} a_{1} + x_{2} a_{2} + \dots + x_{n} a_{n}

. This is how you think. Solving

A x = b

asks: can

b

be built from the columns?

Theorem 1.3.1 · Consistency

A x = b

is consistent if and only if

b

can be written as a linear combination of the column vectors of

A

In §3.6 this gets the upgrade it deserves: consistent ⟺ $b$ lies in the column space of $A$ .

Definition · Linear combination

For vectors

a_{1}, \dots, a_{n}

R^{m}

and scalars

c_{1}, \dots, c_{n}

, the vector

c_{1} a_{1} + \dots + c_{n} a_{n}

is a linear combination. Everything in Chapter 3 is built from this one idea.

Matrix multiplication

Definition

A

m \times n

and

B

n \times r

, then

A B

is the

m \times r

matrix with entries

c_{ij} = a_{i} b_{j} = k = 1 \sum n a_{ik} b_{k j} .

Equivalently, column by column: $A B = (A b_{1}, A b_{2}, \dots, A b_{r})$ . Each column of the product is $A$ acting on the corresponding column of $B$ .

Worked example

$A B$ and $B A$ are not the same animal

0/3 steps

Compute both products for

A = 321 - 2 4 - 3, B = (- 2 4 1136)

Transpose

Definition · Transpose

A^{T}

is the

n \times m

matrix with

b_{j i} = a_{ij}

: row

j

A^{T}

is column

j

A

. A square matrix is symmetric if

A^{T} = A

∞Connection

Symmetry is a Chapter 5 concept in disguise.

A^{T} A

is always symmetric — that's the matrix at the heart of the normal equations in §5.3, and its symmetry is why least squares behaves so well.

Check yourself

A

3 \times 4

. The system

A x = b

is consistent exactly when:

T / F

A

3 \times 5

and

B

5 \times 3

, then both

A B

and

B A

are defined and both are

3 \times 3

T / F

A

is any

m \times n

matrix, then

A^{T} A

is symmetric.

1.3Matrix Arithmetic

What you must be able to do

Two ways to see $A x$

Matrix multiplication

$A B$ and $B A$ are not the same animal

Transpose

Check yourself

1.3Matrix Arithmetic

What you must be able to do

Two ways to see Ax

Matrix multiplication

AB and BA are not the same animal

Transpose

Check yourself

Two ways to see $A x$

$A B$ and $B A$ are not the same animal