Seyed Masoud Hosseini · Overview · Study log · Weekly summaries · Ideas · Search · Transcript · RSS feed
Deep Learning Systems · Lecture 25 of 25 · 2:36
Deep Learning Systems Online Course Teaser
Study guide
What this lecture covers
This is a promotional teaser, not a lecture. Instructors Zico Kolter and Tianqi Chen introduce the Deep Learning Systems course (CMU 10-414) and announce that it will be offered free and online in Fall 2022. It gives no technical content itself but explains what the course covers and who it's for.
The teaser describes building needle, a small Python-based deep learning framework built entirely from scratch, without relying on NumPy for its final implementation. It states the course covers building an automatic differentiation toolkit, a neural network library, CPU and GPU linear algebra backends, and standard models such as convolutional networks, recurrent networks, and Transformers.
Key ideas
- Instructors: Zico Kolter (CMU faculty, Chief Scientist in AI at Bosch Research) and Tianqi Chen (CMU faculty, Chief Technologist at OctoML, lead developer of XGBoost, MXNet, and Apache TVM).
- needle: the small deep learning framework students build from scratch throughout the course, comparable in spirit to a minimal PyTorch or TensorFlow.
- Course scope: automatic differentiation, a neural network library, CPU and GPU linear algebra backends, and implementations of convolutional networks, recurrent networks, and Transformers.
- No NumPy in the final library: by the end of the course, the linear algebra operations themselves are implemented by students rather than delegated to NumPy.
- Free and open: the course runs as a normal CMU class but is also made available online at no cost, including lectures, materials, forums, homework, and a final project.
- Prerequisites: proficiency in Python and C++, plus some background in machine learning or deep learning; the first lectures and assignment are suggested as a self-assessment.
- Enrollment: interested learners are directed to dlsyscourse.org to fill out an enrollment form.
Before you watch
- No prior course content is required to understand this teaser, since it is an introduction to the course itself.
Check your understanding
- What deep learning framework do students build over the course, and what is notable about its final implementation compared to typical frameworks?
- What are the two suggested prerequisites for taking this course?
- Where can someone interested in enrolling go to sign up?
Vocabulary
- teaser (noun)
- A short video made to introduce or promote something.
This video is a teaser for the course, not a full lecture. - instructor (noun)
- A person who teaches a course.
Two instructors co-teach this course. - faculty (noun)
- The teaching staff at a university.
Both instructors are CMU faculty members. - chief scientist (noun)
- A senior researcher who leads scientific work at a company.
One instructor is a chief scientist in AI at a research lab. - chief technologist (noun)
- A senior person who guides the technical direction of a company.
The other instructor is chief technologist at a startup. - framework (noun)
- A set of pre-built tools used to build software of a certain kind.
Needle is a small deep learning framework. - from scratch (phrase)
- Built from nothing, without relying on ready-made tools.
The framework is built from scratch in this course. - linear algebra (noun)
- The branch of math dealing with vectors, matrices, and their operations.
The course builds its own linear algebra backend. - backend (noun)
- The underlying system that performs the real computation behind an interface.
Students build CPU and GPU backends. - convolutional network (noun)
- A neural network built around convolution operations, often used for images.
The course covers building convolutional networks. - recurrent network (noun)
- A neural network designed to process sequences step by step.
Recurrent networks are covered later in the course. - Transformer (noun)
- A neural network architecture based on self-attention, common in language models.
The course includes building a Transformer. - delegate (verb)
- To pass a task to another system instead of doing it yourself.
The final library does not delegate math to NumPy. - proficiency (noun)
- A good level of skill in something.
Proficiency in Python and C++ is a prerequisite. - prerequisite (noun)
- Something required before starting a course or task.
Basic machine learning knowledge is a prerequisite. - self-assessment (noun)
- A way of judging your own readiness or ability.
The first assignment works as a self-assessment. - enrollment (noun)
- The process of signing up for a course.
Enrollment happens through the course website. - at no cost (phrase)
- Free, without any payment required.
The course materials are offered at no cost. - final project (noun)
- A larger piece of work completed at the end of a course.
Students complete a final project by the end of the course. - forum (noun)
- An online space where people discuss and ask questions.
The course provides a forum for student questions.
Chapters
- 0:00 Introduction
- 0:22 Course Announcement
- 0:35 Course Summary
- 1:39 All For Free
- 1:56 Who Should Take This Course
- 2:17 How To Join
From the YouTube description
Introduction to our online Deep Learning Systems course, info at http://dlsyscourse.org.
The online version of the course starts **September 13, 2022**. Enroll for free by going to our web site and filling out the enrollment form.