Deep Learning Systems Online Course Teaser

What this lecture covers

This is a promotional teaser, not a lecture. Instructors Zico Kolter and Tianqi Chen introduce the Deep Learning Systems course (CMU 10-414) and announce that it will be offered free and online in Fall 2022. It gives no technical content itself but explains what the course covers and who it's for.

The teaser describes building needle, a small Python-based deep learning framework built entirely from scratch, without relying on NumPy for its final implementation. It states the course covers building an automatic differentiation toolkit, a neural network library, CPU and GPU linear algebra backends, and standard models such as convolutional networks, recurrent networks, and Transformers.

Key ideas

• Instructors: Zico Kolter (CMU faculty, Chief Scientist in AI at Bosch Research) and Tianqi Chen (CMU faculty, Chief Technologist at OctoML, lead developer of XGBoost, MXNet, and Apache TVM).
• needle: the small deep learning framework students build from scratch throughout the course, comparable in spirit to a minimal PyTorch or TensorFlow.
• Course scope: automatic differentiation, a neural network library, CPU and GPU linear algebra backends, and implementations of convolutional networks, recurrent networks, and Transformers.
• No NumPy in the final library: by the end of the course, the linear algebra operations themselves are implemented by students rather than delegated to NumPy.
• Free and open: the course runs as a normal CMU class but is also made available online at no cost, including lectures, materials, forums, homework, and a final project.
• Prerequisites: proficiency in Python and C++, plus some background in machine learning or deep learning; the first lectures and assignment are suggested as a self-assessment.
• Enrollment: interested learners are directed to dlsyscourse.org to fill out an enrollment form.

Before you watch

• No prior course content is required to understand this teaser, since it is an introduction to the course itself.

Check your understanding

1. What deep learning framework do students build over the course, and what is notable about its final implementation compared to typical frameworks?
2. What are the two suggested prerequisites for taking this course?
3. Where can someone interested in enrolling go to sign up?