Lecture 1: Introduction and Logistics

What this lecture covers

This opening lecture of Deep Learning Systems (CMU 10-414) answers two questions: why study deep learning systems at all, and how the course runs. It sets up the rest of the series, where you'll build a small PyTorch-like library called needle from scratch, covering automatic differentiation, neural network architectures, and hardware-efficient implementations.

There's no math or code here. After watching, you'll know the course's structure, its prerequisites, and what you're expected to produce by the end: a working deep learning library you wrote yourself.

Key ideas

• needle: the "necessary elements of deep learning" library students build across the homeworks, structured loosely like PyTorch.
• Systems vs. usage: knowing how deep learning systems work internally (not just calling PyTorch) lets you write more efficient code and do better research.
• Automatic differentiation and gradient-based optimization: the two core algorithmic ideas the lecture argues underlie nearly every deep learning advance shown in the intro examples.
• Library adoption drove the boom: Kolter argues that easy-to-use Python autodiff frameworks like Keras, TensorFlow, and PyTorch, not any single research breakthrough, most explain deep learning's rapid growth in interest after 2015.
• Course structure: four homeworks build needle incrementally (autodiff, then an NN library, then CPU/GPU backends, then architectures like CNNs, RNNs, and transformers), plus a group final project extending needle and a course forum.
• Prerequisites: systems programming (basic C++), linear algebra, calculus (derivatives, not integrals), and prior exposure to machine learning.
• Grading: homeworks are entirely code-based and auto-graded locally; 80%+ average plus a final project earns a certificate of completion (no CMU credit for the public version).

Walkthrough

Why deep learning systems matter (3:07)

Kolter opens with a tour of landmark results - AlexNet, AlphaGo, StyleGAN, GPT-3, AlphaFold, and Stable Diffusion - to establish that deep learning already works remarkably well. He's careful to say this isn't a history of the field or an attempt to assign credit for who did what first. He also points out that several of these breakthroughs, like DeOldify and the timm image-model library, came from very small teams, to argue that the field is still open to individual contributors, not just large companies.

How library adoption drove the boom (14:25)

Using a Google Trends chart of interest in "deep learning" since the late 2000s, Kolter shows that public interest stayed flat through the 2012 AlexNet moment and only took off after Keras, TensorFlow, and PyTorch were released around 2015-2016. He argues the biggest driver of adoption was the availability of easy-to-use, Python-based automatic differentiation libraries, not any single algorithmic breakthrough. As an illustration, he contrasts co-instructor Tianqi Chen's 2012 experience - 44,000 lines of code and six months to build a working ConvNet - with today's equivalent, roughly 100 lines and a few hours, thanks to modern frameworks.

Three reasons to study systems, not just use them (21:35)

Kolter gives three reasons to take the course rather than just use existing libraries. First, if you want to build or contribute to deep learning frameworks, you need to understand how they work, since the field is still evolving (he points to JAX's recent rise as evidence it isn't settled). Second, and the reason he stresses most for practitioners, understanding the internals lets you write more efficient and effective code, especially for research on new architectures. Third, he simply finds the underlying algorithms compelling: despite PyTorch and TensorFlow being millions of lines of code, the core ideas are extremely simple and a full-featured library can be written in under 2,000 lines.

Meet the instructors (29:48)

Kolter introduces himself (CMU faculty since 2012, chief scientist in AI research at Bosch, work on adversarial robustness and implicit layers) and his co-instructor Tianqi Chen, describing Chen's role as original developer of XGBoost, a lead developer of MXNet, and a founding developer of Apache TVM. He also flags that this is the first time the course is offered publicly online, so expect bugs and hiccups in the assignments and logistics.

Prerequisites and course logistics (38:57)

The lecture lays out what you should already know: basic systems programming in C++ (enough to write and debug a matrix multiplication routine, no advanced language features needed), linear algebra and derivative-level calculus, some familiarity with mathematical proofs, and prior exposure to machine learning. Kolter suggests trying homework zero - implementing softmax regression and a manual two-layer network - as a self-check on readiness.

Course components, grading, and forum rules (44:03)

The course has four parts: video lectures (recorded live, mostly in one take), four programming homeworks that build needle's autodiff engine, neural network library, CPU/GPU backends, and architectures in sequence, a group final project that must extend needle itself (not just implement an architecture in PyTorch), and a course forum. Anyone can watch the lectures on YouTube, but submitting homeworks requires registering for the course. Homeworks run and grade locally against reference solutions rather than executing on a remote server. Kolter closes by asking students to use the forum respectfully - sharing snippets to help each other is fine, but posting full solutions undermines the point of doing the assignments yourself.

Before you watch

• No prior lecture needed; this is the first in the series.
• Helpful to already know basic machine learning concepts, since the course assumes you're not learning ML itself here.
• If unsure about readiness, the lecture suggests trying homework zero (softmax regression and a manual two-layer network) as a self-check.

Check your understanding

1. What two algorithmic ideas does Kolter say underlie essentially all deep learning architectures?
2. According to the lecture, what factor most closely tracks the timing of deep learning's surge in popularity after 2015?
3. What is the needle library, and how do students build it over the course?
4. What are the four main components of the course, and how is the final project different from the homeworks?