Lecture 12: Visualizing and Understanding

What this lecture covers

After several lectures building architectures for classification, detection, and segmentation, this lecture turns to a different question: what is actually happening inside a trained convolutional network? It surveys a range of techniques, from simply visualizing learned filters to using gradients on the input image itself, that make a network's internal computations more interpretable.

The lecture moves from the simplest tools (looking at first-layer weights, nearest neighbors in feature space) to more involved ones (occlusion experiments, saliency maps, guided backpropagation, gradient-ascent image synthesis) and ends with two techniques built on the same gradient-based machinery, DeepDream and neural style transfer, that are as much about generating striking images as about understanding networks. After watching, you should be able to describe several ways to probe what a CNN layer or neuron responds to, and explain how a gram matrix captures texture for style transfer.

Key ideas

• First-layer filter visualization: because the first convolutional layer takes an inner product directly with image pixels, its weights can be viewed as small images, and they consistently show oriented edges and opposing colors.
• Nearest neighbors in feature space: images that are far apart in raw pixels can be close together in the network's last hidden layer, suggesting that layer captures semantic content.
• t-SNE: a nonlinear dimensionality-reduction technique used to compress high-dimensional last-layer features into two dimensions for visualization.
• Maximally activating patches: for a chosen neuron, finding which image patches (within that neuron's receptive field) produce the highest activation across a dataset reveals what it responds to.
• Occlusion and saliency maps: masking image regions and tracking the change in class probability, or computing the gradient of a class score with respect to input pixels, both highlight which pixels matter most for a prediction.
• Gradient ascent image synthesis: starting from a fixed trained network, you can update the pixels of an input image (instead of the weights) to maximize a class score or neuron value, usually with a regularizer that keeps the result looking natural.
• Fooling images: gradient ascent can change an image's predicted class with an almost imperceptible pixel change, revealing that networks rely on cues humans do not perceive the same way.
• Gram matrix: an average outer product of feature vectors across spatial positions in a layer that discards spatial layout but captures texture statistics, used for texture synthesis and style transfer.

Walkthrough

Visualizing the first layer's filters (5:33)

The lecture starts with the simplest available visualization: since the first convolutional layer computes an inner product directly against image pixels, its filters can be displayed as small RGB images. Across AlexNet, ResNet, and DenseNet, these first-layer filters almost always show oriented edges and opposing colors, echoing the classic finding about the early human visual system covered earlier in the course. The same trick applied to a second layer's weights is shown to be uninterpretable, because those filters connect to the output of the previous layer rather than to raw pixels, motivating the more indirect techniques that follow.

Understanding the last layer: nearest neighbors and t-SNE (13:27)

Rather than the first layer, this section studies the last hidden layer (a 4096-dimensional feature vector before the final classification layer). Computing nearest neighbors in this feature space, instead of in raw pixel space, retrieves images that look visually different but are semantically similar, evidence that the network has learned something like semantic structure even though nothing in the classification loss explicitly encourages it. t-SNE is then introduced as a way to compress that 4096-dimensional space down to two dimensions for visualization, and applied both to MNIST pixels and to ImageNet features, revealing rough semantic clusters.

Intermediate features: activations and maximally activating patches (22:21)

For intermediate convolutional layers, visualizing activation maps (rather than weights) turns out to be more informative: one slice of AlexNet's conv5 feature map, for instance, activates strongly on human faces in a live webcam demo. The lecture also describes recording, for a chosen neuron, which image patches from a large dataset produce the highest activation, then sorting and displaying those patches. Patches from an early layer tend to show simple structures like colored circles or curved edges, while patches from a higher layer show larger, more complex structures such as faces or camera parts, reflecting the larger receptive fields deeper in the network.

Occlusion and saliency: which pixels matter (29:11)

The occlusion experiment (from Zeiler and Fergus) systematically masks a small region of the input image, replacing it with the mean pixel value, and records how the predicted class probability changes as the occluded region slides across the image; a large drop indicates the network relied heavily on those pixels. Saliency maps offer a faster alternative: the gradient of the predicted class score with respect to input pixels gives, in one backward pass, a rough map of which pixels matter most, and can even be paired with a segmentation algorithm to outline objects without any pixel-level labels, though this works markedly worse than a network trained explicitly for segmentation.

Gradient ascent, guided backpropagation, and fooling images (34:20)

Guided backpropagation modifies how gradients flow through ReLU units, keeping only positive gradients, to produce sharper images showing which input pixels drive a given neuron. The lecture then generalizes this into gradient ascent: freezing the network's weights and instead updating the pixels of an image to maximize a class score or neuron value, typically starting from a blank or noisy image and adding a regularizer (an L2 penalty, periodic blurring, or clipping small values) so the result looks more like a natural image rather than adversarial noise. This same gradient-based approach produces fooling images, where a tiny, visually imperceptible pixel change causes a network to confidently predict a completely different class, a phenomenon the lecture flags as a topic for a later dedicated lecture.

DeepDream, feature inversion, and style transfer (51:52)

DeepDream amplifies whatever features a chosen layer already detects in an image by setting the gradient at that layer equal to its own activation and repeatedly updating the image, producing surreal, dream-like results (famously dog-like shapes, since roughly 200 of ImageNet's 1,000 classes are dog breeds). Feature inversion reconstructs an image from its feature vector at a given layer to show how much low-level detail that layer retains: early layers preserve pixels almost exactly, while deeper layers keep spatial layout but lose fine texture and color. Texture synthesis captures a texture's statistics using a gram matrix, an averaged outer product of feature vectors across spatial locations in a layer, and generates a new image whose gram matrix matches it via gradient ascent. Combining a content image's feature reconstruction loss with a style image's gram matrix loss yields style transfer, and the lecture notes that this slow, per-image optimization can be replaced by training a fast feed-forward network that applies a fixed style in a single forward pass.

Before you watch

• Be comfortable with CNN architecture basics (convolution, ReLU, pooling, fully-connected layers) and how backpropagation computes gradients, both from earlier lectures.
• Recall the transpose convolution and downsampling/upsampling ideas from the previous lecture on segmentation, since the fast style-transfer network reuses that structure.
• Familiarity with gradient descent for training will help since several techniques here invert it into gradient ascent on the image.

Check your understanding

1. Why does visualizing the raw weights of the first convolutional layer give useful intuition, while doing the same for a deeper layer does not?
2. Describe two different ways covered in the lecture to determine which pixels of an input image most influenced a network's classification decision.
3. What is a gram matrix, and why does it discard spatial information while still being useful for texture synthesis?
4. What happens when you perform gradient ascent to maximize a class score without any regularizer, and why does adding one help produce more interpretable images?
5. Why are fooling images considered surprising, and what does that suggest about what a trained network is actually relying on?