Lecture 3: Orthonormal Columns Give Q'Q = I

What this lecture covers

This lecture is a tour of orthogonal matrices, the matrices Strang calls the queens of linear algebra. Starting from the defining property Q^T Q = I, he shows why orthogonal matrices preserve vector length, then works through a gallery of examples: rotation and reflection matrices, Householder reflections, Hadamard-style plus-minus-one matrices, and Haar wavelet matrices.

The lecture closes by connecting orthogonal matrices to eigenvectors, previewing the discrete Fourier transform as the eigenvector matrix of a permutation matrix. After watching, you should be able to verify whether a matrix is orthogonal, explain why orthogonal matrices don't cause numerical overflow, and recognize several standard families of orthogonal matrices by construction.

Key ideas

• Q^T Q = I for orthonormal columns: this single identity captures both that each column has length 1 and that different columns are perpendicular.
• Length preservation: ||Qx|| = ||x|| for any vector x, proven directly from Q^T Q = I, which is why orthogonal matrices avoid overflow or underflow in numerical computation.
• Rotations and reflections: the 2x2 matrix with columns [cos θ, sin θ] and [-sin θ, cos θ] rotates the plane; swapping the sign on the second column instead gives a reflection matrix.
• Householder reflections: H = I - 2uu^T for a unit vector u is both symmetric and orthogonal, and is used in numerical linear algebra as an alternative to Gram-Schmidt.
• Hadamard-style matrices: square matrices of plus and minus ones with orthogonal columns can be built recursively (doubling in size), though whether one exists for every size divisible by four remains an open conjecture.
• Haar wavelets: an orthogonal matrix built from rescaled patterns of 1, -1, and 0 that captures averages and differences at multiple scales.
• Orthogonal eigenvectors: the eigenvectors of a symmetric matrix, and also of an orthogonal matrix, are automatically orthogonal; the eigenvectors of a permutation matrix give the discrete Fourier transform.

Walkthrough

Why Q'Q = I preserves length (0:01)

Strang defines Q as a matrix with orthonormal columns and shows Q^T Q = I follows directly from the columns being unit length and mutually perpendicular. He then proves ||Qx|| = ||x|| for any x by expanding (Qx)^T(Qx) and using Q^T Q = I, explaining that this length preservation is why numerical algorithms favor orthogonal matrices wherever possible.

Rotation and reflection matrices (3:03)

Working with 2x2 examples, Strang shows that [cos θ, -sin θ; sin θ, cos θ] rotates the whole plane by angle theta, while changing the sign pattern to [cos θ, sin θ; sin θ, -cos θ] produces a reflection matrix instead, with a determinant of -1. He illustrates both by tracking where the standard basis vectors land.

Householder reflections (15:21)

For a unit vector u, Strang defines H = I - 2uu^T and verifies by direct calculation that H is both symmetric and orthogonal (H^2 = I). He notes these matrices are practically important in numerical linear algebra as an alternative to Gram-Schmidt for producing orthogonal matrices.

Hadamard and Haar wavelet matrices (20:26)

Strang builds Hadamard-style matrices of plus and minus ones recursively, doubling from a 2x2 block to 4x4, 8x8, and beyond, and mentions the open conjecture that such matrices exist for every size divisible by four. He then constructs the Haar wavelet matrix, whose columns represent averages and differences at successively finer scales, noting that Haar's simple construction predates the general theory of wavelets by decades, with major later advances credited to Ingrid Daubechies.

Orthogonal eigenvectors and the Fourier matrix (35:43)

Strang closes by noting that eigenvectors of a symmetric matrix, and separately of an orthogonal matrix, are always orthogonal to each other. He builds the eigenvectors of a permutation matrix explicitly using powers of i, showing this produces the columns of the discrete Fourier transform matrix, and cautions that checking orthogonality for complex vectors requires the complex conjugate rather than a plain dot product.

Before you watch

• Watch Lectures 1 and 2 first: this lecture assumes familiarity with column space, rank, and the factorizations introduced there.
• Basic comfort with complex numbers (i, powers of i, complex conjugates) is needed for the closing Fourier matrix discussion.

Check your understanding

1. Why does Q^T Q = I imply that Qx has the same length as x?
2. What is the geometric difference between the rotation matrix and the reflection matrix built from cos θ and sin θ?
3. How does a Householder matrix H = I - 2uu^T manage to be both symmetric and orthogonal?
4. What pattern lets Hadamard-style matrices be built recursively, and what remains unresolved about their existence?
5. Why must checking orthogonality of complex eigenvectors use the complex conjugate rather than a standard dot product?