Lecture 2: Multiplying and Factoring Matrices

What this lecture covers

This lecture continues the review of linear algebra highlights by naming the five factorizations that recur throughout the course: A = LU, A = QR, symmetric S = QΛQ^T, the singular value decomposition, and A = CR from the previous lecture. Strang uses the column-times-row view of matrix multiplication introduced last time to show how each of these factorizations breaks a matrix into rank-one pieces.

The second half turns to the fundamental theorem of linear algebra: the four fundamental subspaces of a matrix (column space, row space, and their two null spaces), their dimensions, and the fact that the row space and null space are orthogonal to each other. By the end you should be able to name the five factorizations, explain what makes the spectral theorem work, and state the dimensions of the four fundamental subspaces for a given matrix shape and rank.

Key ideas

• Five key factorizations: A = LU (elimination), A = QR (orthogonal columns via Gram-Schmidt), symmetric S = QΛQ^T (eigenvectors and eigenvalues), the singular value decomposition A = UΣV^T (works for every matrix, even rectangular ones), and A = CR.
• Spectral theorem as a sum of rank-one pieces: S = QΛQ^T breaks into λ1 q1 q1^T + λ2 q2 q2^T + ... + λn qn qn^T, each term a symmetric rank-one matrix built from one eigenvector.
• Orthonormal eigenvectors: for a symmetric matrix, the eigenvalues are real and the eigenvectors can be chosen orthonormal, forming a complete set.
• Elimination as peeling off rank-one pieces: each step of elimination removes a rank-one piece built from the current pivot row and column, leaving a smaller matrix to factor next, which builds up A = LU.
• The four fundamental subspaces: column space C(A), row space C(A^T), null space N(A), and left null space N(A^T).
• Dimension counts: for an M x N matrix of rank R, the row and column spaces both have dimension R; the null space has dimension N - R; the left null space has dimension M - R.
• Orthogonality of row space and null space: Ax = 0 says x is orthogonal to every row of A, so the row space and null space sit at 90 degrees to each other inside R^N.

Walkthrough

Five factorizations and the spectral theorem (1:03)

Strang lists the five factorizations by name and letter convention, then focuses on the symmetric case S = QΛQ^T. Splitting S into rank-one pieces λ1 q1 q1^T + λ2 q2 q2^T + ... and checking it against Sq1 = λ1 q1 confirms the formula reproduces the correct eigenvalues and eigenvectors, using the orthonormality of the q vectors to cancel every term but one.

Elimination as column-times-row peeling (17:24)

Working a 2x2 example, Strang reinterprets ordinary elimination, subtracting a multiple of one row from another to produce A = LU, as repeatedly pulling a rank-one piece (built from the current pivot's column and row) out of the matrix and leaving a smaller matrix behind. Each stage of elimination corresponds to peeling off one more rank-one layer, so the final L and U are assembled from these successive column-times-row pieces.

The four fundamental subspaces (27:45)

Strang introduces the fundamental theorem of linear algebra: for an M x N matrix A of rank R, the column space and row space both have dimension R, while the null space N(A) (solutions to Ax = 0) has dimension N - R and the left null space N(A^T) has dimension M - R. He shows the null space is closed under addition and scalar multiplication, which is what makes it a genuine vector space, and argues from counting independent equations why its dimension must be N - R.

The geometry: row space and null space are orthogonal (42:14)

Using a small 2x3 example, Strang finds a basis for the null space directly and shows that each null space vector is orthogonal to each row of A, since Ax = 0 is exactly the statement that every row's dot product with x is zero. This means the row space and null space sit at right angles inside R^N, with dimensions R and N - R adding up to N; the column space and left null space have the same relationship inside R^M.

Before you watch

• Watch Lecture 1 first: the column space, rank, and the factorization A = CR introduced there are used directly here.
• Comfort with matrix-vector multiplication Ax as a combination of columns is assumed.

Check your understanding

1. What are the five factorizations Strang names, and which matrices does each apply to?
2. How does the check Sq1 = λ1 q1 confirm the spectral theorem's rank-one decomposition?
3. How does viewing elimination as peeling off rank-one pieces lead to the factorization A = LU?
4. For an M x N matrix of rank R, what are the dimensions of all four fundamental subspaces?
5. Why must the row space and null space of a matrix be orthogonal to each other?