Lecture 1: Introduction to Convolutional Neural Networks for Visual Recognition

What this lecture covers

This opening lecture answers two questions: why computer vision matters, and how the field arrived at convolutional neural networks. It traces vision from its evolutionary origins through early computer vision research to the 2012 ImageNet breakthrough, then switches to a walkthrough of what the course itself will teach.

There are no technical prerequisites for this lecture. It sets the stage for the rest of the course, which builds toward implementing convolutional neural networks from scratch. After watching, you'll understand why image classification is the course's central problem, how CNNs relate to decades of earlier work, and what the syllabus, assignments and grading look like.

Key ideas

• Computer vision as the study of visual data: the lecture frames the field around the sheer scale of visual data online, citing an estimate that video made up roughly 80% of internet traffic by 2017.
• Hubel and Wiesel's cat experiments: 1950s-60s electrophysiology showing that the primary visual cortex first detects simple oriented edges before building up to more complex structures, which shaped later computational models.
• David Marr's vision pipeline: a 1970s idea that vision proceeds from a "primal sketch" of edges, to a "2.5D sketch" of surfaces and depth, to a full 3D model.
• Feature-based recognition: work like David Lowe's SIFT features showed that matching small, distinctive parts of an object is more robust than matching a whole object under changing viewpoint and lighting.
• ImageNet: a dataset of millions of labeled images across thousands of categories, built to give recognition algorithms enough data to avoid overfitting and to benchmark progress.
• The 2012 breakthrough: a convolutional neural network (later called AlexNet) cut the ImageNet classification error rate from around 25% to 16% in one year, launching the deep learning era in vision.
• CNNs are not new: similar architectures existed in the 1990s (Yann LeCun's digit-recognition network); what changed by 2012 was available compute (GPUs) and available labeled data.
• Image classification as the course's core task: CS231n uses image classification as the foundation for related problems such as object detection and image captioning.

Walkthrough

Why computer vision matters (0:07)

The instructor opens by noting the course's rapid growth and frames computer vision as the study of visual data. He argues the field is urgent because of the huge and growing volume of images and video online, comparing unanalyzed visual data to "dark matter" that is abundant but hard to observe directly. He also places computer vision as an interdisciplinary field touching physics (optics), biology and psychology (how brains see), and computer science and engineering (building the systems).

A history of vision, from animal eyes to cameras (7:34)

Fei-Fei Li's pre-recorded segment traces biological vision back roughly 540 million years, when the sudden evolution of eyes triggered a rapid increase in animal species diversity, sometimes called "evolution's Big Bang." From there the lecture moves to mechanical vision: the 1600s camera obscura, and Hubel and Wiesel's mid-20th-century electrophysiology experiments on cat brains, which found that the visual cortex responds first to simple oriented edges.

From block world to feature-based object recognition (13:44)

The lecture covers early computer vision research: Larry Roberts's "Block World" thesis reducing scenes to simple geometric shapes, the 1966 MIT Summer Vision Project's overly optimistic goal of solving vision in one summer, and David Marr's staged model of visual processing (primal sketch, 2.5D sketch, full 3D representation). It then covers 1970s-80s work representing objects as combinations of generalized cylinders or parts, image segmentation research from Malik and Shi, real-time face detection using AdaBoost (which reached commercial cameras by 2006), and feature-based approaches like SIFT and histogram of oriented gradients that made recognition more robust to viewpoint and lighting changes.

Benchmarks and the rise of ImageNet (28:39)

As datasets like PASCAL VOC enabled measurable progress on object recognition, the Stanford and Princeton team built ImageNet, a dataset assembled by downloading billions of internet images organized by the WordNet hierarchy and labeled via crowdsourcing. The resulting ImageNet Large-Scale Visual Recognition Challenge tested algorithms on 1.4 million images across 1,000 classes. The lecture shows the error rate on this benchmark falling steadily from 2010 to 2015, with the sharpest single-year drop, from about 25% to 16%, happening in 2012.

The 2012 breakthrough and why CNNs succeeded when they did (34:49)

The 2012 ImageNet winner was a convolutional neural network from Geoffrey Hinton's group (Alex Krizhevsky and Ilya Sutskever), later known as AlexNet. Every subsequent ImageNet winner has been a neural network, and networks have grown deeper each year, from AlexNet's roughly seven or eight layers to 19-layer VGG and GoogLeNet in 2014, to 152-layer Residual Networks in 2015. The lecture stresses that CNNs are not a 2012 invention: Yann LeCun's 1998 digit-recognition network used a similar architecture. What changed between the 1990s and 2012 was the availability of much greater compute (helped by GPUs) and much larger labeled datasets.

Course overview: image classification and related tasks (35:51)

Justin Johnson introduces the course's central task, image classification, assigning an image one label from a fixed set of categories, and explains that despite its simplicity it underlies many real applications. The course also covers related problems built on the same tools: object detection (drawing bounding boxes around multiple objects) and image captioning (generating a natural-language description of an image). The lecture notes that current vision systems, however capable, still fall well short of the rich, contextual understanding people show when interpreting even a single image, illustrated with examples of dense scene description and humor recognition.

Course logistics and prerequisites (50:02)

The lecture closes with practical details: the teaching staff, communication through Piazza, an optional deep learning textbook, and the course's emphasis on implementing CNNs from scratch in Python while also introducing tools like TensorFlow and PyTorch. Grading includes three problem sets, a midterm, and a team course project. Stated prerequisites are strong Python skills, calculus, linear algebra, and some prior exposure to computer vision and machine learning, though key concepts will be reintroduced as needed.

Before you watch

• No prior course material is required; this is the first lecture.
• Basic familiarity with what a neural network is will help the later sections land, though the lecture does not assume it.
• If you plan to follow the assignments later in the course, note the stated expectation of solid Python skills and basic calculus and linear algebra.

Check your understanding

1. What two developments between the 1990s and 2012 explain why convolutional neural networks suddenly became state of the art?
2. How did David Marr's model of vision (primal sketch, 2.5D sketch, 3D model) reflect what Hubel and Wiesel found in cat brains?
3. Why did the field build ImageNet, and what problem was it meant to solve for machine learning algorithms at the time?
4. What is the difference between image classification, object detection, and image captioning, and how are they related?
5. According to the lecture, in what ways do human visual understanding still exceed what computer vision systems could do in 2017?