Lecture 1: The Column Space of A Contains All Vectors Ax

What this lecture covers

Gilbert Strang opens 18.065 by reviewing the linear algebra that the rest of the course leans on, starting from the most basic operation: multiplying a matrix by a vector. He pushes past the row-by-row "dot product" view most students learn first, toward seeing Ax as a combination of the columns of A. That shift in viewpoint is the thread that runs through the whole lecture.

From there the lecture builds up column space, rank, and a first matrix factorization A = CR, using it to prove that the number of independent columns always equals the number of independent rows. It closes with a second look at matrix multiplication itself, showing AB as a sum of rank-one pieces rather than a grid of dot products. After this lecture you should be able to describe the column space of a small matrix, find a basis for it by inspection, and explain why row rank and column rank must match.

Key ideas

• Ax as a combination of columns: multiplying A by x is best understood as x1 times the first column plus x2 times the second, and so on, not as a set of separate dot products.
• Column space C(A): the set of all possible outputs Ax as x ranges over every vector; for a 3x3 matrix this can be all of R3, a plane, or a line, depending on the matrix.
• Rank: the number of independent columns, which equals the dimension of the column space.
• Rank-one matrices: a matrix like a column vector times a row vector, uv^T, is the basic building block of matrices whose column space is a single line.
• The factorization A = CR: C holds the independent columns of A taken left to right, and R records how every column of A is built from those independent columns.
• Column rank equals row rank: the same factorization A = CR, read as combinations of the rows of R, shows the rows of A are built from just as many independent rows as there are independent columns.
• Matrix multiplication as a sum of outer products: AB can be computed as the sum over k of (column k of A) times (row k of B), each a rank-one piece, giving the same total number of multiplications as the row-times-column method.

Walkthrough

Matrix times vector, the column way (4:03)

Strang starts with a 3x3 example and contrasts two ways of computing Ax: the row-by-row dot-product method taught first in most courses, and the column combination method he wants students to adopt instead. Seeing Ax as x1 times column one plus x2 times column two plus x3 times column three treats the matrix as a single object rather than a grid of numbers, which sets up the rest of the lecture.

The column space and rank (8:11)

Taking all possible vectors x and collecting every output Ax produces the column space C(A). Strang shows that for a random 3x3 matrix this fills all of R3, but for a matrix whose third column is the sum of the first two, the column space collapses to a plane, and for a matrix whose columns are all multiples of each other it collapses further to a line. The number of independent columns needed to describe that space is the rank of the matrix.

Rank-one matrices and finding a basis (13:15)

A matrix whose column space is a single line factors as a column vector times a row vector, uv^T, a rank-one matrix. Strang then works through the earlier plane example column by column, keeping a column in the basis only if it is not a combination of the ones already kept. This left-to-right process gives a working method for finding a basis for the column space by inspection.

The factorization A = CR and the rank theorem (20:27)

Strang builds C from the independent columns of A and R from the coefficients needed to reconstruct every column of A from those in C, so A = CR. Reading the same equation as combinations of the rows of R shows that the rows of A are spanned by only as many rows as there are columns in C. This gives a proof that the row rank of a matrix always equals its column rank, a fact Strang treats as the first major theorem of the course.

Sampling a large matrix and multiplying A times B (35:53)

For a matrix too large to inspect directly, Strang notes that Ax for a random vector x is always in the column space, so averaging over many random x vectors gives a practical way to sample a huge matrix's column space. He then returns to matrix multiplication itself, showing that AB can be computed either as row-times-column dot products or as a sum of rank-one pieces, column k of A times row k of B. Counting multiplications both ways gives the same total, M x N x P for an M x N matrix times an N x P matrix, confirming the two views are equivalent.

Before you watch

• Comfort with basic matrix-vector multiplication (the row-by-row dot product method) makes it easier to follow the switch to column combinations.
• No prior 18.065 background is assumed; Strang treats this as a review of 18.06 material presented from a new angle.

Check your understanding

1. Why is Ax better understood as a combination of columns rather than a set of dot products?
2. For a 3x3 matrix whose third column equals the sum of the first two, what is the column space, and why?
3. How does the factorization A = CR prove that row rank equals column rank?
4. How can random vectors be used to sample the column space of a very large matrix?
5. Why do the row-times-column and column-times-row methods of computing AB require the same number of multiplications?